247 lines
13 KiB
Markdown
247 lines
13 KiB
Markdown
# 2026-02-13 (Friday)
|
||
|
||
## GPU Research Session (4:14-4:48 AM)
|
||
|
||
Johan explored GPU options for local AI/CoS. Key progression:
|
||
|
||
1. **Started with RTX A6000 48GB ($4,599)** — too expensive for the value
|
||
2. **RTX 8000 48GB (~$2,000)** — same VRAM, older/slower, better price
|
||
3. **RTX 3090 24GB (~$850)** — faster than RTX 8000 but only 24GB
|
||
4. **Tradeoff crystallized:** 3090 = fast but limited VRAM, RTX 8000 = slower but can run 70B models
|
||
5. **Johan's concern:** slow assistant = "I'll do it myself" — speed matters for adoption
|
||
6. **Real motivation revealed:** NOT cost savings — he wants **persistent memory/consistent CoS**. Tired of amnesia.
|
||
7. **Cloud GPU rental** (RunPod, Vast.ai) works for periodic LoRA training without buying hardware
|
||
8. **Conclusion direction:** Better memory pipeline (RAG + nightly distillation) > buying GPU hardware
|
||
- Distillation/memory work is cheap model work (Qwen, K2.5, Gemini Flash)
|
||
- Opus stays for live conversation judgment
|
||
- No hardware purchase needed — fix the software/memory problem instead
|
||
|
||
### Key insight from Johan
|
||
"It's not about money! It's about consistent memory" — the amnesia problem is his #1 frustration with AI assistants.
|
||
|
||
## Qwen2.5-32B Assessment
|
||
- Compared to Opus 4: B/B+ vs A+ (solid mid-level vs senior engineer)
|
||
- Compared to Opus 3.5: closer but still noticeable gap
|
||
- No local model today is good enough for full autonomous CoS role
|
||
- 6 months from now: maybe (open-source improving fast)
|
||
|
||
## Alex Finn Post (@AlexFinn/2021992770370764878)
|
||
- Guide on running local models via LM Studio — 1,891 likes
|
||
- Good for basics but focused on cost savings, not memory persistence
|
||
|
||
## Cloudflare Agent Content Negotiation
|
||
- Cloudflare adding `Accept: text/markdown` at the edge for AI agents
|
||
- **Added to inou TODO** (`/home/johan/dev/inou/docs/TODO.md`)
|
||
- Relevant: inou should be agent-first, serve structured markdown to AI assistants
|
||
- Competitive differentiator vs anti-bot health platforms
|
||
|
||
## Email Triage
|
||
- 1 new email: Amazon shipping (EZVALO motion sensor night lights, $20.32, arriving today)
|
||
- Updated delivery tracker, trashed email
|
||
- **MC performance issue:** queries taking 15-16 seconds consistently — needs investigation
|
||
|
||
## RTX 5090 Scam
|
||
- Johan found $299 "RTX 5090" on eBay — zero feedback seller, obvious scam. Warned him off.
|
||
|
||
## Webchat Bug
|
||
- Johan's message got swallowed (NO_REPLY triggered incorrectly), he had to resend
|
||
|
||
## Cron Jobs → Kimi K2.5 on Fireworks
|
||
- Switched 7 cron jobs from Opus to Kimi K2.5 (`fireworks/accounts/fireworks/models/kimi-k2p5`)
|
||
- K2.5 Watchdog, claude-usage-hourly, git-audit-hourly, dashboard usage, git-audit-daily, update check, weekly memory synthesis
|
||
- Qwen 2.5 32B deprecated/removed from Fireworks — only Qwen3 models remain
|
||
- Qwen3 235B MoE had cold-start 503s (serverless scaling to zero) — unreliable
|
||
- K2.5 stays warm (popular model), ~9s runs, proven in browser agent
|
||
- Fireworks provider registered in OpenClaw config with two models: K2.5 (primary) + Qwen3 235B (backup)
|
||
|
||
## Fireworks Blog Post
|
||
- Fireworks published blog about OpenClaw + Fireworks integration
|
||
- Pitch: use open models for routine tasks (10x cheaper), Opus for judgment
|
||
- Validates our exact setup
|
||
|
||
## Shannon VPS — New Credentials (from Hostkey/Maxim)
|
||
- IP: 82.24.174.112, root / K_cX1aFThB
|
||
- **DO NOT disable password login** until Johan confirms SSH key access (lesson learned from Feb 11 lockout)
|
||
- Task: Install Shannon (KeygraphHQ/shannon) and test against inou portal ONLY
|
||
- Server ID: 53643, HostKey panel: https://panel.hostkey.com/controlpanel.html?key=639551e73029b90f-c061af4412951b2e
|
||
|
||
## Fire Tablet Alert Dashboard (new project)
|
||
- Johan doesn't see Signal alerts reliably — wants a spare Fire tablet (Fully Kiosk) as alert display
|
||
- Requirements: clock, calendar, notification push with sound ("modest pling")
|
||
- Two approaches discussed: standalone web page (preferred) vs Home Assistant integration
|
||
- Johan OK with me coding it or using HA
|
||
- Plan: simple HTML dashboard on forge, SSE for push alerts, Fully Kiosk loads URL
|
||
|
||
## GPU Purchase Decision
|
||
- No GPU purchase yet — persistent memory problem better solved with software (RAG + nightly distillation)
|
||
- If buying: RTX 8000 48GB (~$2K) best option for fine-tuning/70B models
|
||
- Cloud GPU (RunPod/Vast.ai) viable for periodic LoRA training
|
||
|
||
## MC Performance Issue
|
||
- Message Center queries taking 15-16 seconds consistently — needs investigation
|
||
|
||
## Alert Dashboard — Port Conflict Fixed
|
||
- Subagent built alert-dashboard (Node.js/Express, SSE, analog clock, calendar, alert feed)
|
||
- Initially deployed on port 9201 — **WRONG, that's DocSys's port**
|
||
- Moved to port **9202**, restored DocSys on 9201
|
||
- Service: `alert-dashboard.service` (systemd user, enabled)
|
||
- Source: `/home/johan/dev/alert-dashboard/`
|
||
- API: GET /, GET /api/alerts, POST /api/alerts, GET /api/alerts/stream (SSE)
|
||
- Fully Kiosk URL: `http://192.168.1.16:9202`
|
||
|
||
## Shannon VPS — Setup Progress
|
||
- SSH key from forge works ✅ (root@82.24.174.112)
|
||
- Password login: root / K_cX1aFThB — **LEFT ENABLED per instructions**
|
||
- Repo cloned to /opt/shannon
|
||
- Docker build started (still building when subagent finished)
|
||
- TODO: Check build completion, run portal test against inou.com
|
||
|
||
## Kaseya Device Policy Change (IMPORTANT)
|
||
- CISO Jason Manar announced: only Kaseya-issued IT-managed devices on corporate network
|
||
- Personal/BYO devices → BYO network only, no VPN access
|
||
- Rolling out "starting tomorrow" (Feb 14) over coming weeks
|
||
- Johan currently uses personal Mac Mini for EVERYTHING (Kaseya + inou)
|
||
- Has a Kaseya XPS14 laptop he hates
|
||
- **Recommended:** Request a MacBook Pro (CTO-level ask), keep Mac Mini for inou on BYO network
|
||
- Johan is upset about this — impacts his entire workflow
|
||
|
||
## Cron Job Fixes
|
||
- git-audit-hourly timeout bumped 60s → 120s (K2.5 needs more time for git operations)
|
||
- claude-usage-hourly had stale Qwen3 235B session — will self-correct on next run
|
||
- K2.5 Watchdog hit session lock error — transient from concurrent subagent spawns
|
||
|
||
## Email Triage — 7:13 AM (cron)
|
||
|
||
**Mail bridge restarted** — /messages/new was hanging, health OK. Restarted service, recovered.
|
||
|
||
**Trashed (9):**
|
||
- TradingView plan expired (johan@) — Basic plan now, renew if needed
|
||
- Fleur du Mal "Bianca Lace Pink Lady" — color variant, not new collection
|
||
- LinkedIn "7 new messages" notification
|
||
- Trump Media DJT digital token update (corporate press release)
|
||
- Zillow ×3 (851 Brightwaters now listed at $7.25M! + price cut alert + 2 results)
|
||
- Glamuse lingerie marketing
|
||
- Amazon EZVALO shipping (already tracked)
|
||
|
||
**Archived (2):**
|
||
- Z.ai GLM-5 announcement → posted to dashboard news. Claims Opus 4.5-level coding. Johan has dev account (Lite tier, GLM-5 not available yet for Lite).
|
||
- Dialectica consulting (Ryan Rousse Grossman) — IT Disaster Recovery Market, paid consulting. Archived as cold outreach.
|
||
|
||
**Kept in Inbox (1 new):**
|
||
- Cronometer Gold renewal — Feb 20, Mastercard ending 2208. FYI for Johan.
|
||
|
||
**Already in Inbox (previously triaged, awaiting Johan's action):**
|
||
- Neel Madan reply re: Sophia's MRI
|
||
- Diana Geegan listing agreement
|
||
- 4× dotloop signed docs (listing agreement, broker disclosure, seller disclosure, flood disclosure)
|
||
- Dotloop "review 5 documents" request
|
||
- Hostkey support ×2 (CS-391069)
|
||
- Tailscale trial ended
|
||
- Frontier auto-pay confirmation
|
||
|
||
**Notable:** 851 Brightwaters now live on Zillow at **$7,250,000** (up from $7M net sheet). Listed by Keller Williams.
|
||
|
||
## Email Triage — 8:34 AM (cron)
|
||
|
||
**Trashed (5):**
|
||
- TradingView "80% off" Valentine's marketing
|
||
- TradingView "plan expired" (already noted at 7am)
|
||
- HOSTKEY CS-391069 ticket closed (x2 duplicate)
|
||
- Z.ai GLM-5 announcement (already posted to dashboard at 7am)
|
||
|
||
**Archived (7):**
|
||
- Frontier auto-pay $59.99 (2/13, account *0185, card *2269)
|
||
- Dotloop signed confirmations x4 (listing agreement, broker disclosure, seller disclosure, flood disclosure — 851 Brightwaters)
|
||
- Dotloop "review 5 documents" (already completed/signed)
|
||
- Diana Geegan listing agreement PDF (ingested to docs)
|
||
- Tailscale trial ended (downgrade to Personal or pick paid plan)
|
||
|
||
**Kept in Inbox (2):**
|
||
- **Dr. Neel Madan** — confirmed Sunday 2PM call re: Sophia's MRI. "Perfect. Talk to you then."
|
||
- Dr. Neel Madan earlier message (thread context)
|
||
|
||
## Email Triage — 9:13 AM (cron)
|
||
|
||
**Kept in Inbox (action required):**
|
||
- **Dr. Neel Madan** — Confirmed call **Sunday 2PM** re: Sophia's Dec 31 MRI. Johan replied 1:37am, Neel said "Talk to you then." CRITICAL.
|
||
- **Belastingdienst** — Corporate tax filing (vennootschapsbelasting) reminder for 2025, entity ending ***871. Deadline in Mijn Belastingdienst Zakelijk.
|
||
- **Cronometer** — Gold subscription renews Feb 20, Mastercard *2208.
|
||
|
||
**Trashed (1):**
|
||
- Nordic Pure — President's Day marketing
|
||
|
||
**Archived (1):**
|
||
- Older Dr. Madan thread (id:155, superseded by id:161)
|
||
|
||
## Verizon Purchase (document inbox)
|
||
- Johan switched to Verizon: 4 new lines, 4 iPhones (3x iPhone 17, 1x iPhone 16 Plus)
|
||
- All devices $0/mo with 36-month promo credits
|
||
- Monthly estimate: $170.97
|
||
- His number 727-225-2475 porting from Mint Mobile
|
||
- New numbers: 727-225-3810, 727-307-3952, 727-358-1196
|
||
- Paid $232.40 today (taxes/fees)
|
||
- Receipt stored: documents/store/f281445ed5af1426.pdf
|
||
- Record: documents/records/bills/verizon-purchase-2026-02-13.md
|
||
- Note: Mikhail was on Mint Mobile too — this confirms the family is moving to Verizon
|
||
|
||
### 8:22 PM — Document Inbox: Real Estate PDFs
|
||
6 Brightwaters listing documents arrived in ~/documents/inbox/ at 20:09:
|
||
- Exclusive_Right_of_Sale_Listing_Agreement_ERS-21tb.pdf
|
||
- Brightwaters_MLS_ACT.pdf
|
||
- Seller_Disclosure_Residential.pdf
|
||
- Sellers_Property_Disclosure_SPDR-4x.pdf
|
||
- SellerDisclosureUpdate.pdf
|
||
- Office_Exclusive_No_Entry_Into_MLS.pdf
|
||
Doc processor is inactive. Will flag for Johan when he wakes (~10:15pm).
|
||
|
||
## RCA & Architecture Overhaul (10:30 PM - midnight)
|
||
|
||
### Promotion: Chief of Staff
|
||
Johan promoted James from assistant to **Chief of Staff**. Key principles:
|
||
- Strategic partner, not task runner
|
||
- Memory protection is job #1 — write it down or lose it
|
||
- Right model for the right job (Opus for judgment, K2.5 for grunt work)
|
||
- Autonomous subagent delegation — spin up freely, name them, use appropriate models
|
||
- **Never ask "who is X?" if I've encountered it before** — my memory is my reputation
|
||
|
||
### Johan's Core Philosophy (MEMORIZE)
|
||
- **"Always attack problems at their source. I hate workarounds. They will bite me in the ass tomorrow."**
|
||
- Don't band-aid. Fix the root cause.
|
||
- If something is triggering noise, fix the trigger — don't filter the noise downstream.
|
||
- Example: instead of triaging emails in OC (workaround), embed K2.5 triage directly IN Message Center (source fix).
|
||
|
||
### Architecture Changes Made
|
||
1. **K2.5 Watchdog cron killed** — 288 phantom sessions/day, never caught anything, monitoring a dead agent
|
||
2. **Session cleanup** — 2,514 orphaned files + 320 stale session keys removed (2,843 → 17 files)
|
||
3. **Email webhook moved** — from main session → mail agent (stops email triage polluting conversation)
|
||
4. **K2.5 triage embedded in Message Center** — MC now calls Fireworks directly, reads prompt from file (hot-reloadable), only webhooks on escalation. Logs all actions to tablet dashboard.
|
||
5. **Daily reset moved** — 4am → 9pm (aligned with Johan's first sleep block)
|
||
6. **Nightly Maintenance cron** (9pm) — memory flush + OS/Claude Code/OpenClaw updates + session cleanup, all in one shot
|
||
7. **Cron consolidation** — killed Email Triage Summary (AM/PM), Update Check, K2.5 Watchdog, git-audit-daily-full. Merged claude-usage jobs. 350 sessions/day → ~43.
|
||
8. **Context pruning enabled** — `cache-ttl` mode, trims old tool results after 5min. Dramatically reduces compaction pressure.
|
||
|
||
### Tools Reminder
|
||
- **bird for X/Twitter** — always. Don't use web_fetch for tweets.
|
||
- bird wrapper at ~/clawd/scripts/bird
|
||
|
||
### Config State After Changes
|
||
- session.reset.atHour: 21 (9pm ET)
|
||
- contextPruning.mode: "cache-ttl", ttl: "5m"
|
||
- hooks.messages → sessionKey: "mail" (was "main")
|
||
- MC triage prompt: ~/clawd/config/email-triage-prompt.md
|
||
|
||
### Memory System Discussion (11:54 PM)
|
||
- Discussed LoRA vs file-based memory — LoRA would bake patterns into weights, eliminating the "read files to pretend I remember" gap
|
||
- Johan proposed SQLite + embeddings as alternative to .md files
|
||
- Tested existing memory_search — it already works well (Gemini embeddings, 0.80+ scores for Shannon queries across sessions)
|
||
- **Key insight:** The problem isn't search quality — it's that I don't USE memory_search consistently before responding
|
||
- Added MANDATORY memory_search rule to AGENTS.md: "Search before speaking, no exceptions"
|
||
- Johan's framing: "I will write the number down if I think it is important" — hybrid approach (explicit + auto-extraction)
|
||
- OpenClaw has `maxInjectedChars` config but doesn't auto-inject memory results per-turn — it's a tool I must call
|
||
- Future consideration: auto-inject memory search results into system prompt before each turn (gateway-level)
|
||
|
||
### Late Night Session Summary
|
||
This was the most productive infrastructure session we've had. Everything stems from Johan's philosophy: **attack problems at their source, not downstream.**
|
||
- Started with RCA on continuity loss
|
||
- Ended with a fundamentally restructured system: email triage in MC, session management aligned to schedule, context pruning active, cron rationalized, memory discipline enforced
|