johan
/
clawd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
clawd/memory/security-scans/2026-03-22.md

9.9 KiB
Raw Permalink Blame History

Security Posture Scan — 2026-03-22

Scan conducted twice: 09:00 AM ET and 14:37 ET (this file reflects both) Conducted by: James (weekly cron job)

AM Scan Summary (09:00 ET)

Host Status Issues
forge (192.168.1.16) ⚠️ WARNING 3 findings (zombie+rogue server killed live)
james-old (192.168.1.17) ⚠️ WARNING RDP still open (known), xrdp running
staging (192.168.1.253) ✅ CLEAN Matches baseline
prod (192.168.100.2) ❌ UNREACHABLE SSH key not installed
caddy (192.168.0.2) ⚠️ WARNING New user hans:1002 — needs confirmation
zurich (82.22.36.202) ✅ CLEAN High brute force volume (normal for VPS)

PM Scan Summary (14:37 ET)

Host Status Issues
forge (192.168.1.16) ⚠️ WARNING OC gateway high CPU (83%), VNC unauth'd, hans key unconfirmed
james-old (192.168.1.17) ❌ UNREACHABLE SSH timeout (was accessible this morning)
staging (192.168.1.253) ✅ CLEAN ClickHouse high CPU (expected), all services healthy
prod (192.168.100.2) ❌ UNREACHABLE SSH auth failure (key not installed)
caddy (192.168.0.2) ⚠️ WARNING rsyslogd+journald CPU storm; hans:1002 still unconfirmed
zurich (82.22.36.202) ✅ CLEAN +32 bans since AM scan, all hardening intact

Forge (192.168.1.16) — ⚠️ WARNING

AM Findings (Actions Taken)

[FIXED] Zombie bash process (PID 3673859) — 99.9% CPU for ~5 days

  • /bin/bash -c openclaw logs --follow | head -30 ... — spinning log follow loop
  • Killed. Confirmed gone.

[FIXED] Rogue python3 http.server on port 8000 (LAN-bound)

  • Unexpected listener, no legitimate service
  • Killed. Port confirmed closed.

PM Findings (Ongoing)

[WARNING] openclaw-gateway at 83% CPU (PID 1374638)

  • Running since 04:41 today, accumulated 496 CPU-minutes
  • High but may be normal during heavy agentic work / active sessions
  • Monitor: if sustained at >80% for hours without active sessions, investigate

[INFO] opencode process at 52% CPU (PID 1062817, pts/14)

  • Started Mar 21, 1033 hours CPU time — long-running dev session
  • Owner: johan, legitimate dev tool

[INFO] fireworks-proxy on 127.0.0.1:18484

  • PID 1060741: /usr/bin/python3 /home/johan/.local/bin/fireworks-proxy
  • localhost only, legitimate API proxy

[KNOWN] x11vnc on port 5900 (all interfaces)

  • PID 3936577, running since Mar 18
  • VNC without visible password flags in cmdline — authentication status unverified
  • Baseline: not in baseline ports list. Needed for headed Chrome.
  • Recommendation: Restrict to LAN or verify VNC password is set.

[INFO] hans@vault1984-hq key still in authorized_keys

  • Added 2026-03-08, marked "pending confirmation" in baseline
  • Has NOT been removed. Still awaiting Johan's confirmation.

[INFO] Port 8888 dev server (clavitor) — GONE in PM scan

  • Was present in AM scan. No longer listening. Clean.

Users

✅ johan:1000, scanner:1001 — matches baseline

Login History

✅ All from 192.168.1.14 (Johan's Mac) or 100.114.238.41 (Tailscale). Clean.

Failed Logins

✅ None (LAN host, not brute-forced)

Crontab (PM check)

✅ All entries are expected:

  • backup-forge.sh (nightly 3am)
  • claude-usage-check.sh (hourly)
  • ddns-update.sh (every 5 min)
  • health-push.sh (every minute)
  • vault1984-twitter-drip.sh (Mar 18-19 scheduled tweets, past dates)

SSH Hardening

⚠️ Cannot verify without sudo (user-level only — known limitation)

UFW

❌ NOT installed (known deficiency — relying on router/network controls)

fail2ban

✅ Active

James-Old (192.168.1.17) — ❌ UNREACHABLE (PM scan)

SSH timeout (10s) in PM scan. Was accessible in AM scan (user-level).

Possible causes:

  • Machine asleep/powered off
  • Network issue
  • SSH service crashed

Action needed: Johan to check on james-old. Last known login: Mar 2.

AM findings (carried forward):

  • Port 3389 (RDP/xrdp) running — origin still unknown from baseline
  • UFW/SSH hardening could not be verified (user-level access only)

Staging (192.168.1.253) — ✅ CLEAN

Users

✅ johan:1000 only

SSH Keys

Known keys + johan@inou (informational — not in baseline but legitimate dev device)

Login History

Last login: Mar 1 from 192.168.1.14. Machine rarely accessed.

Listening Ports

✅ All within baseline. Notable:

  • clickhouse (8123/9000), immich (2283), jellyfin (8096), signal-cli (8080)
  • inou services: api (8082), portal (1080), viewer (8765), dbquery (9124)
  • Home Assistant (8123) — overlaps with clickhouse port; both via Docker

Processes

[INFO] ClickHouse at 468% CPU — normal for a multi-core database server under load. Running in Docker (restarted 7 hours ago — fresh start). Healthy.

Docker

✅ All containers healthy:

  • clickhouse (7h up), immich_server (7h, healthy), immich_machine_learning (7h, healthy)
  • signal-cli-rest-api (7 days, healthy), immich_postgres (6 weeks), immich_redis/valkey (6 weeks), jellyfin (6 weeks)

OpenClaw

Not running on staging (was in baseline — likely decommissioned there). No concern.

Prod (192.168.100.2) — ❌ UNREACHABLE

SSH returns "Too many authentication failures" — key not installed for james@forge. Caddy IS connecting to prod (192.168.0.2→192.168.100.2:1080 outbound seen on caddy), so prod is alive.

Action needed: Install james@forge SSH key on prod for future auditing.

Caddy (192.168.0.2) — ⚠️ WARNING

⚠️ NEW: rsyslogd + journald CPU Storm

rsyslogd: 120% CPU / journald: 57.2% CPU

  • On a Raspberry Pi, this is severe. These processes have been running since Mar 13.
  • Total CPU time accumulated: rsyslogd 15,973 minutes, journald 7,610 minutes
  • Indicates a logging loop or log storm (possibly from caddy access logs, fail2ban, or a failing service)
  • Recommendation: Check /var/log/syslog size and caddy access log volume. May need logrotate tuning.
  • Not blocking, but will impact Pi performance and SD card lifespan.

[CARRIED] hans:1002 — Unconfirmed

  • User exists with bash shell and SSH access (key: hans@vault1984-hq)
  • Same fingerprint as hans key in forge's authorized_keys
  • Not in baseline. Needs Johan's confirmation that this was intentional.

Users

⚠️ hans:1002 — unconfirmed (see above) ✅ stijn:1001 — expected (flourishevents web account)

Root SSH Keys

✅ Only james@forge — matches baseline exactly

Login History

✅ No interactive logins since boot (Aug 5, 2025). Clean.

Failed Logins

✅ None (LAN-accessible only, not publicly brute-forced)

Listening Ports

✅ All expected: 22, 80, 443, 40021 (vsftpd), 1984 (caddy proxying vault1984), 2283 (caddy proxying immich)

SSH Hardening

✅ passwordauthentication no, permitrootlogin without-password, pubkeyauthentication yes

UFW

✅ Active. Rules unchanged from AM scan.

fail2ban

❌ Not running (known from baseline — never installed)

TLS Certificate

✅ inou.com cert valid: Mar 5 – Jun 3, 2026 (73 days remaining)

Security Patches

⚠️ linux-image-raspi 6.8.0-1048 security kernel update pending (same as AM scan — not yet applied)

Outbound

✅ tailscaled (normal), SSH from james (192.168.1.16), caddy → 192.168.100.2:1080 (prod proxy)

Zurich (82.22.36.202) — ✅ CLEAN

SSH Brute Force (fail2ban)

  • Total bans since boot: 2,741 (was 2,709 at AM scan — +32 in ~5.5h, normal rate ~6/hour)
  • Currently banned: 4 active bans
  • Recent attempts: ubuntu, susanna, default, sol, shop, admin, harryhaa — all blocked ✅
  • 5 jails active: caddy-kuma, caddy-scanner, sshd, stalwart, vaultwarden ✅

Users

✅ harry:1000, harry-web:1001 — matches baseline exactly

Root SSH Keys

✅ All 5 keys match baseline exactly. No additions or removals.

Login History

Last root logins: Jan 27 from 47.197.93.62 (home IP) — no interactive logins since. ✅ Current connections: SSH from forge (47.197.93.62) — James' tool connections. ✅

Listening Ports

✅ All within baseline: SSH, Stalwart mail (25/143/465/587/993/995/4190), 80/443 (Caddy), 3001 (Kuma)

UFW

✅ Active with 24 rules. Port 3001 (Kuma) IS in UFW allow rules — externally accessible. Note: This is a known issue from baseline. Kuma accessible at zurich.inou.com:3001.

SSH Hardening

✅ passwordauthentication no, permitrootlogin without-password, pubkeyauthentication yes

Security Patches

✅ No pending security updates

Outbound

✅ Tailscale only + SSH inbound from forge. Clean.

Actions Taken This Scan Cycle

  1. [AM] Killed zombie bash log-follow process (PID 3673859) — 5-day 99.9% CPU zombie
  2. [AM] Killed rogue python3 -m http.server 8000 — unexpected LAN-bound listener

Open Items for Johan (Consolidated)

🔴 Critical / Confirm Required

  1. Caddy: hans:1002 user — Unconfirmed since last scan. Has SSH login access. Confirm or remove.
  2. Forge: hans@vault1984-hq SSH key — Still "pending confirmation" since 2026-03-08. Confirm or remove.

🟡 Warnings

  1. Caddy: rsyslogd/journald CPU storm — 120%/57% CPU on Raspberry Pi. Check log volume, potential disk/SD wear. Run: journalctl --disk-usage and du -sh /var/log/syslog*
  2. James-Old: UNREACHABLE in PM scan — Was accessible at 9am. Check if machine is up.
  3. Caddy: Kernel security update — linux-image-raspi 6.8.0-1048 ready to install.
  4. Forge: VNC (x11vnc) on port 5900 — Verify VNC password is set. Restrict to LAN if not needed externally.
  5. Forge: openclaw-gateway at 83% CPU — Monitor. May be normal during heavy agentic sessions.

🔵 Informational / Housekeeping

  1. Prod (192.168.100.2) — Install james@forge SSH key to enable future audits.
  2. Caddy: fail2ban — Still not installed (known from baseline).
  3. James-old: xrdp/RDP (3389) — Still flagged since baseline. Disable if not needed.
  4. Zurich: Port 3001 (Kuma) — Externally accessible via UFW. Consider closing if Caddy proxy is sufficient.