clavitor/clavis/clavis-vault/SPEC-replication-config.md

# Replication Configuration — Mandatory in Commercial Edition

## Principle: Config File, Not Flags

Commercial Edition requires explicit `replication.yaml` configuration file.
No env vars. No optional flags. Replication is core to the commercial value proposition.

## Config File: `/etc/clavitor/replication.yaml`

```yaml
# Commercial Edition replication configuration
# This file is REQUIRED for commercial builds. Without it, vault refuses to start.

# This POP's identity
pop_id: "calgary-01"
region: "north-america"
role: "primary"  # or "backup"

# Backup POP to replicate to (required for primary role)
backup_pop:
  id: "zurich-01"
  url: "https://zurich-01.clavitor.ai"
  auth_token_file: "/etc/clavitor/replication.key"
  
# Inter-POP authentication
# Token is read from file (not inline for security)
auth:
  token_file: "/etc/clavitor/replication.key"
  mtls_cert: "/etc/clavitor/replication.crt"
  mtls_key: "/etc/clavitor/replication.key"

# Replication behavior (optional, defaults shown)
replication:
  batch_size: 100
  max_retries: 5
  # No interval - event-driven only
  
# Control plane for failover coordination (optional but recommended)
control_plane:
  url: "https://control.clavitor.ai"
  token_file: "/etc/clavitor/control-plane.key"
  heartbeat_interval: 60  # seconds
```

## Startup Behavior

### Commercial Edition

```go
// main.go commercial path
cfg, err := edition.LoadReplicationConfig("/etc/clavitor/replication.yaml")
if err != nil {
    log.Fatalf("Commercial edition requires replication.yaml: %v", err)
}

// Validate: primary role requires backup_pop configured
if cfg.Role == "primary" && cfg.BackupPOP.URL == "" {
    log.Fatalf("Primary POP requires backup_pop.url in replication.yaml")
}

edition.SetCommercialConfig(&edition.CommercialConfig{
    ReplicationConfig: cfg,
})
```

**Without config file: vault refuses to start.**

### Community Edition

Ignores replication config entirely. Single-node operation only.

```go
// main.go community path
// No replication config loaded
// No replication worker started
// Works exactly as before
```

## Role-Based Behavior

### Primary POP (e.g., Calgary)

- Accepts client writes
- Replicates to configured backup POP
- Serves normal traffic

### Backup POP (e.g., Zurich)

- Accepts replication pushes from primary
- Rejects client writes with 503
- Serves read-only traffic if promoted
- **Does not replicate further** (prevent cascade)

### Failover (Manual or Control-Plane Managed)

```yaml
# Before failover (Zurich is backup)
role: "backup"
primary_pop:
  id: "calgary-01"
  url: "https://calgary-01.clavitor.ai"

# After failover (promoted to primary)
role: "primary"
backup_pop:
  id: "calgary-01"  # Old primary becomes backup
  url: "https://calgary-01.clavitor.ai"
```

## Security

### Token Files

```bash
# /etc/clavitor/replication.key
# 256-bit random token, base64 encoded
# Generated at POP provisioning time

chmod 600 /etc/clavitor/replication.key
chown clavitor:clavitor /etc/clavitor/replication.key
```

### mTLS (Optional but Recommended)

```bash
# Each POP has unique client cert signed by internal CA
# Presented to backup POP for authentication

# Generate per-POP cert
openssl req -new -key zurich-01.key -out zurich-01.csr
openssl x509 -req -in zurich-01.csr -CA clavitor-intermediate.crt ...
```

## Why Config File > Env Vars

| Aspect | Environment Variables | Config File |
|--------|----------------------|-------------|
| **Visibility** | Hidden in shell/env | Explicit in `/etc/clavitor/` |
| **Validation** | Runtime checks only | Startup validation with clear errors |
| **Structure** | Flat strings | YAML hierarchy, comments |
| **Secrets** | Exposed to all subprocesses | File permissions control access |
| **Versioning** | Not tracked | Can be versioned per POP |
| **Rotation** | Hard to rotate live | Signal HUP to reload |

## Implementation

### New File: `edition/config.go` (Commercial Only)

```go
//go:build commercial

package edition

type ReplicationConfig struct {
    POPID     string `yaml:"pop_id"`
    Region    string `yaml:"region"`
    Role      string `yaml:"role"` // "primary" or "backup"
    
    BackupPOP struct {
        ID            string `yaml:"id"`
        URL           string `yaml:"url"`
        AuthTokenFile string `yaml:"auth_token_file"`
    } `yaml:"backup_pop"`
    
    Auth struct {
        TokenFile string `yaml:"token_file"`
        MTLSCert  string `yaml:"mtls_cert"`
        MTLSKey   string `yaml:"mtls_key"`
    } `yaml:"auth"`
    
    Replication struct {
        BatchSize  int `yaml:"batch_size"`
        MaxRetries int `yaml:"max_retries"`
    } `yaml:"replication"`
}

func LoadReplicationConfig(path string) (*ReplicationConfig, error) {
    data, err := os.ReadFile(path)
    if err != nil {
        return nil, err
    }
    var cfg ReplicationConfig
    if err := yaml.Unmarshal(data, &cfg); err != nil {
        return nil, err
    }
    // Validate
    if cfg.Role != "primary" && cfg.Role != "backup" {
        return nil, fmt.Errorf("role must be 'primary' or 'backup'")
    }
    if cfg.Role == "primary" && cfg.BackupPOP.URL == "" {
        return nil, fmt.Errorf("primary role requires backup_pop.url")
    }
    // Set defaults
    if cfg.Replication.BatchSize == 0 {
        cfg.Replication.BatchSize = 100
    }
    return &cfg, nil
}
```

### Modified: `cmd/clavitor/main.go`

Remove replication flags entirely. Replace with:

```go
// Commercial: Load mandatory replication config
if edition.Current.Name() == "commercial" {
    replCfg, err := edition.LoadReplicationConfig("/etc/clavitor/replication.yaml")
    if err != nil {
        log.Fatalf("Commercial edition requires /etc/clavitor/replication.yaml: %v", err)
    }
    edition.SetCommercialConfig(&edition.CommercialConfig{
        ReplicationConfig: replCfg,
    })
    // ... start replication ...
}
```

## Example POP Provisioning

### Calgary (Primary)

```bash
mkdir -p /etc/clavitor

# Generate replication token
cat > /etc/clavitor/replication.key << 'EOF'
clavitor-pop-v1-a1b2c3d4e5f6...
EOF
chmod 600 /etc/clavitor/replication.key

# Create config
cat > /etc/clavitor/replication.yaml << 'EOF'
pop_id: "calgary-01"
region: "north-america"
role: "primary"
backup_pop:
  id: "zurich-01"
  url: "https://zurich-01.clavitor.ai"
  auth_token_file: "/etc/clavitor/replication.key"
auth:
  token_file: "/etc/clavitor/replication.key"
EOF

# Start vault
./clavitor-commercial
# Output: Commercial edition: primary POP, replicating to zurich-01
```

### Zurich (Backup)

```bash
mkdir -p /etc/clavitor

# Same token (shared secret for inter-POP auth)
cp calgary-01:/etc/clavitor/replication.key /etc/clavitor/replication.key

# Backup config (no backup_pop, acts as replica)
cat > /etc/clavitor/replication.yaml << 'EOF'
pop_id: "zurich-01"
region: "europe"
role: "backup"
auth:
  token_file: "/etc/clavitor/replication.key"
EOF

# Start vault
./clavitor-commercial
# Output: Commercial edition: backup POP, accepting replication from primary
```

## Summary

- **Config file, not env**: `/etc/clavitor/replication.yaml`
- **Mandatory for commercial**: Vault refuses to start without it
- **Explicit role**: Primary or backup, no ambiguity
- **File-based secrets**: `token_file`, `mtls_cert` paths
- **No polling**: Event-driven as designed earlier
- **Control plane optional**: For automated failover