inou/docs/journal-system-2026-02.md

615 lines
20 KiB
Markdown

# Journal System Design
*February 2026*
## Overview
The journal system captures insights, protocols, hypotheses, and observations from LLM conversations about health data, making them available for future conversations and medical decision-making. It solves the "lost insights" problem where valuable reasoning and decisions from conversations are not preserved.
**Core principle:** Continuity of care across LLM conversations.
---
## Motivation
### The Problem
**Current state:**
- User has valuable health conversation with LLM (diagnosis exploration, protocol design, symptom analysis)
- Insights and reasoning are lost when conversation ends
- Next conversation starts from scratch - no context of what was discussed, tried, or decided
- User must re-explain medical history, recent events, and current hypotheses
**Example:**
- Feb 8: Wife discusses Sophia's post-fever symptoms with Opus → realizes hair loss may indicate CSF drainage started → develops hypothesis connecting multiple symptoms
- Feb 15: Husband asks different LLM about Sophia's condition → LLM has no knowledge of the drainage hypothesis or reasoning
- Lost context: Jan 20 Dr. I infusion, fever timeline, symptom connections
### The Solution
**Journal entries provide:**
1. **Cross-conversation memory** - insights from one conversation available in the next
2. **Reasoning preservation** - not just "what" but "why" decisions were made
3. **Timeline tracking** - when things were tried, observed, or decided
4. **Status tracking** - what's active, testing, resolved, or discarded
5. **Context for medical decisions** - doctors/family can see reasoning behind interventions
---
## Journal Entry Types
Journals are **entries** in the entries table with `category = CategoryNote` (or category-specific Type="journal").
### Core Types
| Type | Purpose | Example |
|------|---------|---------|
| `protocol` | Intervention plan | "Serrapeptase suppositories for aqueductal clearing" |
| `hypothesis` | Testable idea | "Post-fever CSF drainage via aqueduct" |
| `observation` | Notable finding | "Neurological improvements after Jan 10 fever" |
| `connection` | Linking data points | "MTHFR C677T affects DHA metabolism → impacts omega-3 protocol" |
| `question` | Open question | "Why does glymphatic clearance favor left-lateral position?" |
| `reference` | Background info | "Dr. Ibraheem's WBC therapy mechanism explained" |
### Entry Fields
```go
type Entry struct {
EntryID string // 16-char hex
DossierID string // who this is about
Category int // CategoryNote (or category-specific)
Type string // protocol, hypothesis, observation, etc.
// Display
Value string // Title (encrypted)
Summary string // 1-2 sentence summary (encrypted)
// Content
Data string // JSON (encrypted): full content, source, metadata
// Context
Tags string // comma-separated: "omega-3,aqueduct,nutrition"
Timestamp int64 // when created
Status int // 0=draft, 1=active, 2=resolved, 3=discarded
// Relationships
ParentID string // optional: link to specific entry
SearchKey string // optional: for filtering (encrypted)
}
```
### Data JSON Structure
```json
{
"source": "opus-4.6",
"conversation_id": "uuid-from-claude-desktop",
"conversation_date": "2026-02-08",
"content": "Full markdown content with sections, reasoning, etc.",
"related_entries": ["entry_id_1", "entry_id_2"],
"reasoning": "Why this matters / how we arrived at this",
"metadata": {
"weight": "30kg",
"age": "9yo",
"any_other_context": "relevant to this entry"
}
}
```
---
## Auto-Capture Modes
Users set journal behavior at MCP connection time or in Portal settings.
### Mode 1: Disabled
```json
"journal_mode": "disabled"
```
- LLM never creates journal entries
- User creates manually via Portal
- LLM can READ journals for context
### Mode 2: Suggest (Default)
```json
"journal_mode": "suggest"
```
- LLM drafts entries silently during conversation
- At conversation end: shows summary with approval prompt
- User can edit/approve/discard before saving
**End-of-conversation prompt:**
```
┌─────────────────────────────────────────┐
│ I've drafted 3 journal entries: │
│ │
│ ☑ HYPOTHESIS: Post-fever drainage │
│ Hair loss + improved tone/eyes may │
│ indicate aqueduct drainage started. │
│ │
│ ☑ PROTOCOL: Serrapeptase suppositories │
│ 5-10mg rectal for aqueduct clearing. │
│ │
│ ☑ OBSERVATION: Neurological gains │
│ Better tone, eye tracking, head hold. │
│ │
│ [Save All] [Review/Edit] [Discard] │
└─────────────────────────────────────────┘
```
### Mode 3: Automatic (Power Users)
```json
"journal_mode": "automatic",
"journal_notify": true // optional: show save indicators
```
- LLM creates entries mid-conversation as key moments happen
- No approval needed
- Optional: subtle notification `💾 Saved: Serrapeptase protocol`
- User can review/edit later via Portal
---
## MCP Integration
### Prompts: family_health_context
**Purpose:** Load available dossiers + announce what data exists (without content)
**MCP Protocol:**
```
Request: prompts/list
Response: [{"name": "family_health_context", "description": "..."}]
Request: prompts/get {"name": "family_health_context"}
Response: {messages: [{role: "user", content: {...}}]}
```
**Content Format:**
```
=== Family Health Context ===
Sophia (3b38234f2b0f7ee6)
- Age: 9yo, Weight: 30kg
- Condition: Hydrocephalus (aqueduct obstruction, dx 12/31/25)
Data available:
• Imaging: 3 studies (most recent: 12/31/25)
• Labs: 45 results (most recent: 11/15/25)
• Genome: 23andMe data available
• Documents: 12 files
• Exercise: tracked
• Nutrition: tracked (G-tube)
Recent journals (last 30 days):
• 2026-02-08: Post-fever aqueductal drainage (hypothesis, observing)
Jan 10 fever may have opened aqueduct. Observing neuro improvements.
• 2026-01-20: Dr. I WBC infusion (intervention, administered)
Autologous therapy given during illness.
• 2026-01-15: Caviar omega-3 protocol (protocol, active)
20-25g daily for SPM production targeting aqueduct inflammation.
Active protocols: 4 interventions tracked
---
Johan (xxx)
- Age: [age], self
Data available:
• Genome: 23andMe data available
• Labs: 12 results (most recent: 10/24/25)
Recent journals: 1 entry (last 30 days)
---
Alena (yyy)
- Age: [age], spouse
Data available:
• Genome: 23andMe data available
• Labs: 8 results (most recent: 09/20/25)
Recent journals: 0 entries
```
### Progressive Disclosure Strategy
**Announce EXISTENCE, not CONTENT** (except journals get brief summaries)
| Data Type | Announcement | Reasoning |
|-----------|--------------|-----------|
| **Imaging** | "3 studies (most recent: 12/31/25)" | Forces LLM to call `list_studies()` when relevant |
| **Labs** | "45 results (most recent: 11/15/25)" | Forces LLM to call `query_entries()` with LOINC code |
| **Genome** | "23andMe data available" | Forces LLM to call `query_genome(gene="MTHFR")` when relevant |
| **Documents** | "12 files" | Forces LLM to explore when needed |
| **Journals** | Title + 1-2 sentence summary | **Exception:** Summaries help LLM decide relevance without fetching each |
**Why this matters:**
- LLMs get lazy with summaries - if given "MTHFR C677T heterozygous" they just repeat it
- Announcing existence forces LLM to fetch fresh data: `query_genome(gene="MTHFR")`
- Journals are different: they're already curated insights, summary IS the point
**Journal summary balance:**
- ✅ Title + date + type + status + 1-2 sentence summary
- ❌ Full content, full reasoning, full protocol details
- LLM gets enough to know IF relevant, fetches full entry WHEN relevant
### Summary Generation
Summaries can come from two sources:
**1. Client-provided (preferred):**
- LLM has full conversation context
- Knows WHY this entry matters (e.g., "for aqueduct inflammation" not just "supplement protocol")
- Summary is contextual and specific
**2. Server-generated (fallback):**
- Uses **Gemini** (already deployed for simple tasks)
- Extracts/generates 1-2 sentence summary from content
- Ensures consistent format/length
- Used when client doesn't provide summary
**Server validation (all summaries):**
```go
// Validate or generate summary
if entry.Summary == "" {
// Generate using Gemini
entry.Summary = generateSummaryWithGemini(entry.Title, entry.Content)
} else {
// Validate client-provided summary
if len(entry.Summary) > 300 {
return errors.New("summary too long (max 300 chars)")
}
}
```
**Gemini prompt for summary generation:**
```
Summarize this health journal entry in 1-2 sentences (max 300 chars).
Focus on: what intervention/observation, for what condition, and key details.
Title: {title}
Type: {type}
Content: {content}
Example good summaries:
- "20-25g daily Beluga/Keta targeting phospholipid-DHA → SPM production for aqueduct inflammation."
- "Jan 10 fever may have opened aqueduct drainage. Observing improved tone, eye tracking, head control."
- "Autologous WBC therapy (Utheline/Ricasin/Notokill + ATP) given during active fever."
Summary:
```
### MCP Tools
```
mcp__inou__list_dossiers()
→ Returns list of accessible dossiers
mcp__inou__get_dossier(dossier_id)
→ Returns dossier info (used internally by prompts/get)
mcp__inou__list_journals(dossier_id, days=30, status=null, type=null)
→ Returns journal summaries (for explicit queries)
mcp__inou__get_journal_entry(dossier_id, entry_id)
→ Returns full journal entry with complete content
mcp__inou__create_journal_entry(
dossier_id,
type, // protocol, hypothesis, observation, etc.
title, // short title
summary, // 1-2 sentences
content, // full markdown
tags, // comma-separated
status, // draft, active, testing, resolved, discarded
related_entries // optional: array of entry IDs
)
→ Creates new journal entry
mcp__inou__update_journal_entry(
dossier_id,
entry_id,
status, // optional: change status
append_note // optional: add update without replacing content
)
→ Updates existing entry
```
---
## API Endpoints
### List Journal Summaries
```
GET /api/v1/dossiers/{dossier_id}/journal?days=30&status=active&type=protocol
Response:
{
"journals": [
{
"entry_id": "abc123",
"type": "protocol",
"title": "Serrapeptase suppositories",
"summary": "5-10mg rectal for aqueduct clearing. Stacks with BPC-157, TB-500.",
"date": "2026-02-08",
"status": "draft",
"tags": ["serrapeptase", "aqueduct", "suppository"]
}
]
}
```
### Get Full Journal Entry
```
GET /api/v1/dossiers/{dossier_id}/journal/{entry_id}
Response:
{
"entry_id": "abc123",
"type": "protocol",
"title": "Serrapeptase suppositories for aqueductal clearing",
"summary": "Adding serrapeptase...",
"content": "# Serrapeptase Protocol\n\n## Rationale\n...",
"date": "2026-02-08",
"status": "draft",
"tags": ["serrapeptase", "aqueduct"],
"related_entries": ["def456"],
"source": "opus-4.6",
"metadata": {
"conversation_url": "https://claude.ai/chat/...",
"weight": "30kg"
}
}
```
### Create Journal Entry
```
POST /api/v1/dossiers/{dossier_id}/journal
Body:
{
"type": "protocol", // required
"title": "Serrapeptase suppositories", // required
"summary": "5-10mg rectal for aqueduct...", // optional - generated with Gemini if not provided
"content": "# Full markdown content...", // required
"tags": ["serrapeptase", "aqueduct"], // optional
"status": "draft", // optional - defaults to "draft"
"related_entries": ["def456"], // optional
"metadata": { // optional
"source": "opus-4.6",
"conversation_url": "https://claude.ai/chat/..."
}
}
Response:
{
"entry_id": "abc123",
"created": "2026-02-08T12:34:56Z",
"summary": "5-10mg rectal for aqueduct clearing" // returned (client-provided or server-generated)
}
```
**Note:** If `summary` is not provided in request, server generates it using Gemini. If provided, server validates length (max 300 chars).
### Update Journal Entry
```
PATCH /api/v1/dossiers/{dossier_id}/journal/{entry_id}
Body:
{
"status": "active" // or append_note: "Update: started 2026-02-10"
}
Response:
{
"updated": "2026-02-08T12:35:00Z"
}
```
---
## Storage Schema
Journals are **entries** with `category = CategoryNote` (or category-specific Type="journal").
**No new tables needed.** Use existing `entries` table:
```sql
-- Journal entry example
INSERT INTO entries (
entry_id, dossier_id, category, type,
value, summary, data, tags,
timestamp, status
) VALUES (
'abc123...', '3b38234f2b0f7ee6', 15, 'protocol',
'Serrapeptase suppositories', -- encrypted
'5-10mg rectal for aqueduct clearing', -- encrypted
'{"source":"opus-4.6","content":"..."}', -- encrypted JSON
'serrapeptase,aqueduct,suppository', -- encrypted
1707408000, 0 -- timestamp, status (0=draft)
);
```
**Status values:**
- 0 = draft
- 1 = active
- 2 = resolved
- 3 = discarded
---
## User Configuration
### Custom Instructions (Claude Desktop)
```
At conversation start, load family health context using the
"family_health_context" prompt from inou MCP.
When discussing family members:
- Look up dossier by name (Sophia, Johan, Alena, etc.)
- Reference their journal summaries for context
- Save insights using journal_mode: automatic
```
### Portal Settings
```
Settings → MCP Access → Journal Behavior
○ Disabled (manual only)
○ Suggest at end (default)
● Automatic (save as we go)
□ Show notifications when entries are saved
```
---
## Use Cases
### Case 1: Reference Conversation
**User:** "Sophia losing hair after being sick. Why?"
**LLM sees from family_health_context:**
- Sophia has hydrocephalus (aqueduct obstruction)
- Recent journal: "Post-fever aqueductal drainage hypothesis (Feb 8)"
- Recent journal: "Dr. I WBC infusion (Jan 20)"
**LLM response:**
> "I see from your recent journals that Sophia had a high fever starting Jan 10, and received Dr. I's infusion on Jan 20 while still sick. You've been tracking a hypothesis that the fever may have opened aqueductal drainage, with observed improvements in tone, eye tracking, and head control. You noted hair loss as one of the adjustment symptoms.
>
> This fits the pattern - the hair loss after 5-day high fever is likely anagen effluvium..."
**Auto-saves:**
```json
{
"type": "observation",
"title": "Hair loss confirmed as post-fever anagen effluvium",
"summary": "Hair loss 4 weeks post-fever confirms anagen damage from 5-day high fever, consistent with neurological repair timeline.",
"status": "active"
}
```
### Case 2: Exploratory Conversation
**User:** "Can I substitute Omega 3 with Beluga caviar?"
**LLM sees from family_health_context:**
- Sophia: 9yo, 30kg, hydrocephalus (aqueduct obstruction)
- Data available: Genome (23andMe)
- No existing caviar protocol journal
**LLM response:**
> "For Sophia's aqueduct inflammation, let me think about this carefully. Let me first check her genome data for variants affecting omega-3 metabolism..."
>
> [calls `query_genome(gene="MTHFR,FADS1,FADS2")`]
>
> "I see she has MTHFR C677T heterozygous, which can reduce methylation efficiency and affect DHA metabolism. Caviar provides phospholipid-bound DHA which crosses the blood-brain barrier more efficiently than triglyceride supplements. For a 30kg child targeting neuroinflammation..."
**Conversation develops protocol → Auto-saves:**
```json
{
"type": "protocol",
"title": "Caviar omega-3 protocol for aqueduct inflammation",
"summary": "20-25g daily Beluga/Keta targeting phospholipid-DHA → SPM production. Replaces AlgaeCal. Based on 30kg weight, MTHFR C677T variant.",
"content": "# Caviar Protocol\n\n## Rationale\nPhospholipid-DHA...",
"tags": ["omega-3", "caviar", "DHA", "SPM", "aqueduct"],
"status": "active",
"related_entries": ["mthfr_variant_id"]
}
```
### Case 3: Protocol Evolution
**User (2 weeks later):** "How's the caviar protocol working?"
**LLM sees:**
- Journal: "Caviar protocol (active, started Feb 8)"
**LLM response:**
> "I see you started the caviar protocol (20-25g daily) on Feb 8. It's been 2 weeks - are you seeing any changes in Sophia's neurological status? Any updates on tone, alertness, or symptoms?"
**User provides update → LLM updates journal:**
```json
{
"entry_id": "abc123",
"status": "active",
"append_note": "Update 2026-02-22: After 2 weeks, parents report improved alertness, sleep quality slightly better. No adverse effects. Continuing protocol."
}
```
---
## Implementation Checklist
### Backend (inou API)
- [ ] `GET /api/v1/dossiers/{id}/journal` - list journal summaries
- [ ] `GET /api/v1/dossiers/{id}/journal/{entry_id}` - get full entry
- [ ] `POST /api/v1/dossiers/{id}/journal` - create entry
- [ ] Accept optional `summary` field in request
- [ ] If summary not provided, generate with Gemini
- [ ] Validate summary length (max 300 chars)
- [ ] Return generated summary in response
- [ ] `PATCH /api/v1/dossiers/{id}/journal/{entry_id}` - update entry
- [ ] Validation: type enum, status enum, required fields
- [ ] Gemini integration for summary generation
- [ ] Prompt template for journal summarization
- [ ] Fallback if Gemini unavailable (extract first sentences)
### MCP Server (inou-mcp)
- [ ] Implement `prompts/list` handler
- [ ] Implement `prompts/get` handler for `family_health_context`
- [ ] Fetch accessible dossiers
- [ ] Fetch journal summaries (last 30 days)
- [ ] Format announcement (existence + brief journal summaries)
- [ ] Add `mcp__inou__list_journals` tool
- [ ] Add `mcp__inou__get_journal_entry` tool
- [ ] Add `mcp__inou__create_journal_entry` tool
- [ ] Add `mcp__inou__update_journal_entry` tool
### Portal
- [ ] Journal list page: `/dossier/{id}/journal`
- [ ] Journal entry detail/edit page
- [ ] Settings: journal_mode preference (disabled/suggest/automatic)
- [ ] Integration with existing dossier sections
### Testing
- [ ] Test auto-capture in "suggest" mode
- [ ] Test auto-capture in "automatic" mode
- [ ] Test family_health_context prompt loads correctly
- [ ] Test progressive disclosure (LLM fetches genome when announced)
- [ ] Test journal summaries provide enough context without being too detailed
- [ ] Test summary generation:
- [ ] Client-provided summary is accepted and validated
- [ ] Missing summary triggers Gemini generation
- [ ] Generated summaries are contextual and concise (1-2 sentences)
- [ ] Summary length validation rejects >300 chars
---
## Design Principles
1. **Continuity over novelty** - preserve reasoning across conversations
2. **Progressive disclosure** - announce existence, force fetching
3. **Lazy prevention** - don't give LLMs summaries of raw data (genome, labs)
4. **Summary exception** - journals get brief summaries because they're curated insights
5. **Gemini for consistency** - server generates summaries with Gemini when not provided, ensuring consistent format/quality
6. **User control** - three modes (disabled/suggest/automatic) for different comfort levels
7. **Minimal schema** - reuse existing entries table, no new tables
8. **Source preservation** - track which LLM, which conversation created each entry
9. **Status tracking** - draft → active → resolved/discarded lifecycle
10. **Relationships** - link journals to relevant data (genome variants, lab results, imaging)
11. **Context for care** - doctors and family can see reasoning behind decisions
---
## Future Enhancements
- **Conversation import** - paste claude.ai conversation URL → auto-extract journal entries
- **Browser extension** - detect medical conversations → one-click save to inou
- **Timeline view** - visualize protocols/hypotheses/observations over time
- **Cross-dossier connections** - link family members' journals (e.g., genetic inheritance)
- **LLM-generated summaries** - auto-summarize long entries for quick reference
- **Version history** - track edits to protocols over time
- **Collaboration** - multiple family members/doctors can add notes to entries