johan
/
dealspace
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
dealspace/SECURITY-SPEC.md

1482 lines
43 KiB
Markdown
Raw Blame History

# Dealspace — Security Architecture Specification
**Version:** 1.0 — 2026-02-28
**Status:** Authoritative. Addresses all P0/P1 gaps identified in SPEC-REVIEW.md.
**Scope:** Authentication, authorization, cryptography, session management, channel security, threat model.
---
## 1. Threat Model
### 1.1 Adversaries
| Adversary | Motivation | Capability | Examples |
|-----------|------------|------------|----------|
| **Nation-state APT** | Economic espionage, strategic intelligence | Unlimited budget, zero-days, supply chain | Acquiring competitor intel, blocking foreign acquisition |
| **Competing bidder** | Deal advantage, price manipulation | Social engineering, insider recruitment | Learning seller's reserve price, sabotaging rival bids |
| **Insider threat** | Financial gain, revenge | Legitimate access, knowledge of systems | Disgruntled IB analyst, seller employee with grudge |
| **Opportunistic attacker** | Financial gain | Automated scanning, credential stuffing | Ransomware operators, data brokers |
| **Activist/hacktivist** | Disruption, exposure | DDoS, data leaks, defacement | Anti-M&A groups, short sellers |
### 1.2 Crown Jewels
**Tier 1 — Catastrophic if leaked:**
- Deal terms (valuation, structure, conditions)
- Financial statements before public filing
- Due diligence findings (legal exposure, hidden liabilities)
- Buyer interest/bid information
- Communication threads revealing negotiation strategy
**Tier 2 — Serious if leaked:**
- Request lists (reveals deal scope and concerns)
- Participant identities (who's involved in the deal)
- Timeline information (deal velocity, closing targets)
- Internal routing (who's responsible for what)
**Tier 3 — Embarrassing but recoverable:**
- System configuration details
- User metadata (login times, session patterns)
- Performance metrics
### 1.3 Top 5 Attack Vectors & Mitigations
| # | Attack Vector | Impact | Mitigation |
|---|--------------|--------|------------|
| 1 | **Email spoofing** — attacker sends email as `cfo@seller.com` to inject false answers | Critical: false data in deal room | DKIM mandatory, channel_participants table, quarantine flow (§5) |
| 2 | **Session hijacking** — stolen JWT used to access deal data | Critical: full account takeover | Short-lived tokens, fingerprint binding, single session (§6) |
| 3 | **Privilege escalation** — seller_member grants themselves seller_admin | High: unauthorized access expansion | Role hierarchy enforcement, audit logging (§3.4) |
| 4 | **Cryptographic key compromise** — master key leaked | Critical: all data exposed | Key versioning, rotation procedure, HSM escrow (§4) |
| 5 | **Insider data exfiltration** — authorized user bulk-downloads deal room | High: deal data leaked to competitors | Rate limiting, watermarking, anomaly detection (§9, SPEC.md §6.2) |
---
## 2. Authentication Flow
### 2.1 JWT Structure
All JWTs use **HMAC-SHA256** (HS256) for signing. The signing key is derived per environment.
```json
{
"header": {
"alg": "HS256",
"typ": "JWT"
},
"payload": {
"sub": "user_id (UUID)",
"iss": "dealspace",
"iat": 1709100000,
"exp": 1709103600,
"jti": "unique_token_id (UUID)",
"sid": "session_id (UUID)",
"fp": "device_fingerprint_hash",
"org": "organization_id",
"roles": ["ib_admin"],
"mfa": true,
"ver": 1
}
}
```
**Field definitions:**
| Field | Purpose | Notes |
|-------|---------|-------|
| `sub` | User identifier | UUID, never email |
| `jti` | Token identifier | For revocation lookup |
| `sid` | Session identifier | Links access + refresh tokens |
| `fp` | Fingerprint hash | HMAC-SHA256 of device characteristics |
| `mfa` | MFA completed | Boolean, must be true for sensitive ops |
| `ver` | Token schema version | For forward compatibility |
### 2.2 Token Lifetimes
| Token Type | Lifetime | Renewal |
|------------|----------|---------|
| Access token | **1 hour** | Via refresh token |
| Refresh token | **7 days** | Sliding window on use |
| MFA challenge | **5 minutes** | One-time use |
| Password reset | **15 minutes** | One-time use |
| Invite token | **72 hours** | One-time use |
### 2.3 Token Refresh Flow
```
Client Server
| |
|-- POST /auth/refresh ------------>|
| { refresh_token, fingerprint } |
| |
| Validate: |
| - refresh_token not revoked |
| - fingerprint matches sid |
| - user still has access |
| |
|<-- 200 { access_token, --------- |
| refresh_token (rotated) } |
| |
| Old refresh_token invalidated |
```
**Refresh token rotation:** Every refresh issues a new refresh token and invalidates the old one. Detects token theft (attacker's stolen token fails on next use).
### 2.4 Token Revocation
Revocation is immediate and enforced at the API gateway.
**Revocation triggers:**
- User logs out
- Password change
- Access grant revoked
- MFA reset
- Admin force-logout
- Anomaly detection trigger
**Implementation:**
```sql
CREATE TABLE revoked_tokens (
jti TEXT PRIMARY KEY,
revoked_at INTEGER NOT NULL,
reason TEXT NOT NULL,
expires_at INTEGER NOT NULL -- cleanup after original expiry
);
CREATE INDEX idx_revoked_expires ON revoked_tokens(expires_at);
```
Every request validates `jti` against `revoked_tokens`. Cache in Redis with TTL matching token expiry to avoid DB hit per request.
### 2.5 MFA — Mandatory for IB Admins
**Requirement:** All users with `ib_admin` or `ib_member` roles MUST have MFA enabled. Login without MFA succeeds but returns `mfa: false` token that can only access `/auth/mfa/*` endpoints.
**TOTP Implementation (RFC 6238):**
```go
type TOTPConfig struct {
Secret []byte // 20 bytes, base32 encoded for user
Algorithm string // "SHA1" (compatibility) or "SHA256" (preferred)
Digits int // 6
Period int // 30 seconds
}
```
**Enrollment flow:**
1. User initiates MFA setup
2. Server generates 160-bit secret, stores encrypted in `users.mfa_secret`
3. Server returns QR code (otpauth:// URI) + recovery codes
4. User scans with authenticator app
5. User enters current TOTP to confirm
6. MFA enabled; recovery codes stored hashed
**Recovery codes:**
- 10 codes, 8 alphanumeric characters each
- Stored as bcrypt hashes
- Each code single-use
- Regenerate all codes if <3 remain
**Verification:**
```go
func VerifyTOTP(secret []byte, code string) bool {
// Accept current window ± 1 (90-second tolerance for clock drift)
for offset := -1; offset <= 1; offset++ {
expected := generateTOTP(secret, time.Now().Add(time.Duration(offset*30)*time.Second))
if subtle.ConstantTimeCompare([]byte(code), []byte(expected)) == 1 {
return true
}
}
return false
}
```
### 2.6 SSO/SAML Integration
Enterprise buyers and large sell-side firms will require SSO. Architecture supports SAML 2.0 federation.
**Database schema:**
```sql
CREATE TABLE sso_providers (
id TEXT PRIMARY KEY,
org_id TEXT NOT NULL REFERENCES organizations(id),
provider_type TEXT NOT NULL, -- "saml" | "oidc"
metadata_url TEXT, -- SAML metadata URL
entity_id TEXT NOT NULL,
certificate TEXT NOT NULL, -- packed: IdP signing cert
acs_url TEXT NOT NULL, -- Assertion Consumer Service URL
enabled INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE TABLE sso_sessions (
session_id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
provider_id TEXT NOT NULL REFERENCES sso_providers(id),
saml_session_id TEXT, -- IdP session for SLO
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL
);
```
**SAML flow:**
1. User clicks "Sign in with [Company] SSO"
2. Server generates AuthnRequest, redirects to IdP
3. IdP authenticates user, returns signed Assertion
4. Server validates signature against stored certificate
5. Server extracts user identity from Assertion
6. If user exists with matching SSO link → issue JWT
7. If user unknown → check if org allows JIT provisioning
8. JIT provisioning: create user with default role, require admin approval for elevated access
**Attribute mapping:**
```go
type SAMLAttributeMap struct {
Email string // e.g., "urn:oid:0.9.2342.19200300.100.1.3"
FirstName string
LastName string
Groups string // optional, for role mapping
}
```
---
## 3. Invitation Flow
### 3.1 Invite Token Structure
Invitations use cryptographically secure, time-limited, single-use tokens.
```go
type Invite struct {
ID string // UUID
Token string // 32-byte random, base64url encoded
TokenHash string // SHA-256 of token (stored, never the raw token)
ProjectID string
WorkstreamID string // optional, null = all workstreams
Role string // assigned at invite time
Email string // target email
InvitedBy string // user ID
ExpiresAt int64 // unix ms, default +72h
AcceptedAt int64 // null until used
RevokedAt int64 // null unless revoked
}
```
**Database schema:**
```sql
CREATE TABLE invites (
id TEXT PRIMARY KEY,
token_hash TEXT NOT NULL UNIQUE,
project_id TEXT NOT NULL REFERENCES projects(id),
workstream_id TEXT REFERENCES entries(entry_id),
role TEXT NOT NULL,
email TEXT NOT NULL,
invited_by TEXT NOT NULL REFERENCES users(id),
expires_at INTEGER NOT NULL,
accepted_at INTEGER,
revoked_at INTEGER,
accepted_by TEXT REFERENCES users(id),
created_at INTEGER NOT NULL
);
CREATE INDEX idx_invites_email ON invites(email);
CREATE INDEX idx_invites_project ON invites(project_id);
```
### 3.2 Invite Generation
```go
func CreateInvite(ctx context.Context, projectID, workstreamID, role, email, inviterID string) (*Invite, error) {
// Validate inviter can grant this role (see §3.4)
if !canGrantRole(ctx, inviterID, projectID, role) {
return nil, ErrInsufficientPrivilege
}
// Generate cryptographically secure token
tokenBytes := make([]byte, 32)
if _, err := rand.Read(tokenBytes); err != nil {
return nil, err
}
token := base64.RawURLEncoding.EncodeToString(tokenBytes)
tokenHash := sha256Hex(tokenBytes)
invite := &Invite{
ID: uuid.New().String(),
Token: token, // returned to caller, NOT stored
TokenHash: tokenHash,
ProjectID: projectID,
WorkstreamID: workstreamID,
Role: role,
Email: strings.ToLower(strings.TrimSpace(email)),
InvitedBy: inviterID,
ExpiresAt: time.Now().Add(72 * time.Hour).UnixMilli(),
CreatedAt: time.Now().UnixMilli(),
}
// Store invite (token_hash, not token)
if err := storeInvite(ctx, invite); err != nil {
return nil, err
}
// Send invite email
if err := sendInviteEmail(ctx, invite); err != nil {
// Log but don't fail — invite is valid, email is best-effort
log.Warn("invite email failed", "invite_id", invite.ID, "error", err)
}
return invite, nil
}
```
### 3.3 Invite Acceptance
```go
func AcceptInvite(ctx context.Context, token string, userID string) error {
tokenHash := sha256Hex([]byte(token))
invite, err := getInviteByTokenHash(ctx, tokenHash)
if err != nil {
return ErrInvalidInvite
}
// Validate invite state
now := time.Now().UnixMilli()
if invite.AcceptedAt != 0 {
return ErrInviteAlreadyUsed
}
if invite.RevokedAt != 0 {
return ErrInviteRevoked
}
if invite.ExpiresAt < now {
return ErrInviteExpired
}
// Verify email matches (if user already exists)
user, err := getUser(ctx, userID)
if err != nil {
return err
}
if !strings.EqualFold(user.Email, invite.Email) {
return ErrEmailMismatch
}
// Grant access atomically with invite acceptance
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
// Mark invite as accepted
if err := markInviteAccepted(ctx, tx, invite.ID, userID, now); err != nil {
return err
}
// Create access grant
access := &Access{
ID: uuid.New().String(),
ProjectID: invite.ProjectID,
WorkstreamID: invite.WorkstreamID,
UserID: userID,
Role: invite.Role,
Ops: defaultOpsForRole(invite.Role),
GrantedBy: invite.InvitedBy,
GrantedAt: now,
}
if err := createAccess(ctx, tx, access); err != nil {
return err
}
return tx.Commit()
}
```
### 3.4 Role Hierarchy & Grant Permissions
Users can only grant roles at or below their level:
```go
var RoleHierarchy = map[string]int{
"ib_admin": 100,
"ib_member": 80,
"seller_admin": 70,
"seller_member": 50,
"buyer_admin": 40,
"buyer_member": 30,
"observer": 10,
}
func canGrantRole(ctx context.Context, granterID, projectID, targetRole string) bool {
granterAccess, err := getAccess(ctx, granterID, projectID)
if err != nil {
return false
}
granterLevel := RoleHierarchy[granterAccess.Role]
targetLevel := RoleHierarchy[targetRole]
// Can only grant roles at or below your level
if targetLevel > granterLevel {
return false
}
// Cross-org restrictions
// IB can grant any role
if strings.HasPrefix(granterAccess.Role, "ib_") {
return true
}
// Sellers can only grant seller roles
if strings.HasPrefix(granterAccess.Role, "seller_") {
return strings.HasPrefix(targetRole, "seller_") || targetRole == "observer"
}
// Buyers can only grant buyer roles
if strings.HasPrefix(granterAccess.Role, "buyer_") {
return strings.HasPrefix(targetRole, "buyer_") || targetRole == "observer"
}
return false
}
```
---
## 4. FIPS 140-3 Cryptography
### 4.1 Validated Module Selection
**Primary:** Google BoringCrypto (FIPS 140-2 validated, 140-3 in progress)
Build Go with BoringCrypto:
```bash
GOEXPERIMENT=boringcrypto go build
```
**Alternative for HSM integration:** AWS CloudHSM or Azure Dedicated HSM for master key storage.
**Verification:**
```go
import "crypto/boring"
func init() {
if !boring.Enabled() {
panic("BoringCrypto not enabled — FIPS compliance requires boringcrypto build")
}
}
```
### 4.2 Key Derivation — HKDF-SHA256
All derived keys use HKDF (RFC 5869):
```go
import "golang.org/x/crypto/hkdf"
func DeriveKey(master []byte, context string, length int) []byte {
// info = domain separator + context
info := []byte("dealspace:" + context)
reader := hkdf.New(sha256.New, master, nil, info)
key := make([]byte, length)
if _, err := io.ReadFull(reader, key); err != nil {
panic(err)
}
return key
}
// Specific derivation functions
func DeriveProjectKey(master []byte, projectID string) []byte {
return DeriveKey(master, "project:"+projectID, 32)
}
func DeriveSearchIndexKey(master []byte) []byte {
return DeriveKey(master, "search-index", 32)
}
func DeriveJWTSigningKey(master []byte) []byte {
return DeriveKey(master, "jwt-signing", 32)
}
```
### 4.3 Blind Indexes (Replacing Deterministic Encryption)
**Problem:** Deterministic encryption enables frequency analysis and known-plaintext attacks.
**Solution:** Blind indexes — HMAC-SHA256 of plaintext, truncated for storage efficiency.
```go
func BlindIndex(key []byte, plaintext string, bits int) string {
mac := hmac.New(sha256.New, key)
mac.Write([]byte(strings.ToLower(strings.TrimSpace(plaintext))))
hash := mac.Sum(nil)
// Truncate to requested bits (must be multiple of 8)
bytes := bits / 8
if bytes > len(hash) {
bytes = len(hash)
}
return base64.RawURLEncoding.EncodeToString(hash[:bytes])
}
// Usage for SearchKey/SearchKey2
func PackEntry(entry *Entry, projectKey, indexKey []byte) error {
// SearchKey: blind index of ref number (e.g., "FIN-042")
if entry.Data.Ref != "" {
entry.SearchKey = BlindIndex(indexKey, entry.Data.Ref, 64) // 64-bit index
}
// SearchKey2: blind index of requester org
if entry.Data.RequesterOrg != "" {
entry.SearchKey2 = BlindIndex(indexKey, entry.Data.RequesterOrg, 64)
}
// Summary and Data use AES-256-GCM (non-deterministic)
summaryJSON, _ := json.Marshal(entry.SummaryData)
entry.Summary = Encrypt(projectKey, summaryJSON)
dataJSON, _ := json.Marshal(entry.Data)
entry.Data = Encrypt(projectKey, dataJSON)
return nil
}
```
**Collision probability:** 64-bit blind index = 2^32 entries before 50% collision probability. For deal platforms: sufficient with monitoring.
### 4.4 AES-256-GCM Encryption
```go
func Encrypt(key, plaintext []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
// 12-byte random nonce (standard for AES-GCM)
nonce := make([]byte, gcm.NonceSize())
if _, err := rand.Read(nonce); err != nil {
return nil, err
}
// Prepend nonce to ciphertext
ciphertext := gcm.Seal(nonce, nonce, plaintext, nil)
return ciphertext, nil
}
func Decrypt(key, ciphertext []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
nonceSize := gcm.NonceSize()
if len(ciphertext) < nonceSize {
return nil, ErrCiphertextTooShort
}
nonce, ciphertext := ciphertext[:nonceSize], ciphertext[nonceSize:]
return gcm.Open(nil, nonce, ciphertext, nil)
}
```
### 4.5 Key Versioning
All encrypted data carries a key version for rotation support:
```sql
ALTER TABLE entries ADD COLUMN key_version INTEGER NOT NULL DEFAULT 1;
```
**Ciphertext format:**
```
[1 byte: version] [12 bytes: nonce] [N bytes: ciphertext] [16 bytes: tag]
```
```go
const CurrentKeyVersion = 1
func EncryptVersioned(key []byte, plaintext []byte) ([]byte, error) {
ciphertext, err := Encrypt(key, plaintext)
if err != nil {
return nil, err
}
// Prepend version byte
versioned := make([]byte, 1+len(ciphertext))
versioned[0] = CurrentKeyVersion
copy(versioned[1:], ciphertext)
return versioned, nil
}
func DecryptVersioned(keys map[int][]byte, ciphertext []byte) ([]byte, error) {
if len(ciphertext) < 1 {
return nil, ErrCiphertextTooShort
}
version := int(ciphertext[0])
key, ok := keys[version]
if !ok {
return nil, ErrUnknownKeyVersion
}
return Decrypt(key, ciphertext[1:])
}
```
### 4.6 Key Rotation Procedure
**Trigger conditions:**
- Master key suspected compromised
- Scheduled rotation (annual minimum)
- Personnel departure (key custodian)
- Compliance requirement
**Rotation process:**
```go
type KeyRotation struct {
ID string
OldVersion int
NewVersion int
StartedAt int64
CompletedAt int64
Status string // "pending" | "in_progress" | "completed" | "failed"
ProjectID string // null = all projects
EntriesTotal int64
EntriesRotated int64
ObjectsTotal int64
ObjectsRotated int64
}
```
**Step 1: Generate new master key**
```bash
# Generate in HSM (preferred) or secure environment
openssl rand -hex 32 > new_master_key.txt
# Store in HSM / Vault / secure key management
```
**Step 2: Derive new project keys**
```go
func RotateProjectKey(ctx context.Context, projectID string, oldMaster, newMaster []byte) error {
oldKey := DeriveProjectKey(oldMaster, projectID)
newKey := DeriveProjectKey(newMaster, projectID)
newVersion := CurrentKeyVersion + 1
// Process entries in batches
batchSize := 100
offset := 0
for {
entries, err := getEntriesByProject(ctx, projectID, batchSize, offset)
if err != nil {
return err
}
if len(entries) == 0 {
break
}
for _, entry := range entries {
// Decrypt with old key
summary, err := Decrypt(oldKey, entry.Summary)
if err != nil {
return fmt.Errorf("decrypt summary: %w", err)
}
data, err := Decrypt(oldKey, entry.Data)
if err != nil {
return fmt.Errorf("decrypt data: %w", err)
}
// Re-encrypt with new key
entry.Summary, _ = Encrypt(newKey, summary)
entry.Data, _ = Encrypt(newKey, data)
entry.KeyVersion = newVersion
if err := updateEntry(ctx, entry); err != nil {
return err
}
}
offset += batchSize
}
return nil
}
```
**Step 3: Rotate object store**
```go
func RotateObjects(ctx context.Context, projectID string, oldMaster, newMaster []byte) error {
oldKey := DeriveProjectKey(oldMaster, projectID)
newKey := DeriveProjectKey(newMaster, projectID)
objects, err := listObjectsByProject(ctx, projectID)
if err != nil {
return err
}
for _, objID := range objects {
data, err := objectStore.Read(objID)
if err != nil {
return err
}
plaintext, err := Decrypt(oldKey, data)
if err != nil {
return err
}
newData, err := Encrypt(newKey, plaintext)
if err != nil {
return err
}
// New object ID (content-addressable)
newObjID := sha256Hex(newData)
if err := objectStore.Write(newObjID, newData); err != nil {
return err
}
// Update references in entries
if err := updateObjectReferences(ctx, projectID, objID, newObjID); err != nil {
return err
}
// Delete old object after references updated
objectStore.Delete(objID)
}
return nil
}
```
**Step 4: Update key version**
```go
// After all entries and objects rotated
CurrentKeyVersion++
// Old key can be archived after grace period (30 days)
```
### 4.7 Key Escrow & Recovery
**Master key custody:** Split across multiple custodians using Shamir's Secret Sharing.
```go
import "github.com/hashicorp/vault/shamir"
func SplitMasterKey(master []byte, shares, threshold int) ([][]byte, error) {
return shamir.Split(master, shares, threshold)
}
func RecoverMasterKey(shares [][]byte) ([]byte, error) {
return shamir.Combine(shares)
}
```
**Recommended configuration:**
- 5 shares, 3 required to reconstruct
- Shares stored in separate secure locations:
- Corporate Vault (CTO)
- Legal escrow (law firm safe deposit)
- Backup HSM (geographically separate)
- CEO secure storage
- Board-designated custodian
**Recovery procedure (documented and tested annually):**
1. Incident declared by authorized personnel
2. Two custodians authenticate to key management portal
3. Each provides their share
4. System reconstructs master key in HSM
5. Rotation triggered immediately (compromised key assumed)
6. Audit log captures all recovery events
---
## 5. Email Channel Authentication
### 5.1 DKIM Verification — Mandatory
All inbound email MUST pass DKIM verification. Failed DKIM = quarantine, never process.
```go
import "github.com/emersion/go-msgauth/dkim"
func VerifyDKIM(rawEmail []byte) (*DKIMResult, error) {
verifications, err := dkim.Verify(bytes.NewReader(rawEmail))
if err != nil {
return nil, err
}
result := &DKIMResult{
Passed: false,
Verifications: make([]DKIMVerification, 0),
}
for _, v := range verifications {
dv := DKIMVerification{
Domain: v.Domain,
Selector: v.Identifier,
Status: v.Err == nil,
Error: "",
}
if v.Err != nil {
dv.Error = v.Err.Error()
} else {
result.Passed = true // At least one valid signature
}
result.Verifications = append(result.Verifications, dv)
}
return result, nil
}
```
### 5.2 Channel Participants Table
Only authorized participants can interact via email. Unknown senders → quarantine.
```sql
CREATE TABLE channel_participants (
id TEXT PRIMARY KEY,
entry_id TEXT NOT NULL REFERENCES entries(entry_id),
channel TEXT NOT NULL, -- "email" | "slack" | "teams"
external_id TEXT NOT NULL, -- email address, slack user ID, teams user ID
user_id TEXT REFERENCES users(id), -- linked internal user, if known
verified INTEGER NOT NULL DEFAULT 0,
added_by TEXT NOT NULL REFERENCES users(id),
added_at INTEGER NOT NULL,
last_used_at INTEGER,
UNIQUE(entry_id, channel, external_id)
);
CREATE INDEX idx_channel_participants_entry ON channel_participants(entry_id);
CREATE INDEX idx_channel_participants_external ON channel_participants(channel, external_id);
```
### 5.3 Inbound Message Flow
```go
func ProcessInboundEmail(ctx context.Context, rawEmail []byte) error {
// Step 1: Parse email
msg, err := mail.ReadMessage(bytes.NewReader(rawEmail))
if err != nil {
return quarantine(ctx, rawEmail, "parse_error", err.Error())
}
// Step 2: DKIM verification (MANDATORY)
dkimResult, err := VerifyDKIM(rawEmail)
if err != nil || !dkimResult.Passed {
return quarantine(ctx, rawEmail, "dkim_failed", "No valid DKIM signature")
}
// Step 3: Extract sender and thread ID
from := msg.Header.Get("From")
senderEmail := extractEmail(from)
inReplyTo := msg.Header.Get("In-Reply-To")
references := msg.Header.Get("References")
// Step 4: Find thread mapping
threadID := findThreadID(inReplyTo, references)
thread, err := getChannelThread(ctx, "email", threadID)
if err != nil || thread == nil {
return quarantine(ctx, rawEmail, "unknown_thread", "No matching thread found")
}
// Step 5: Verify sender is participant
participant, err := getChannelParticipant(ctx, thread.EntryID, "email", senderEmail)
if err != nil || participant == nil {
return quarantine(ctx, rawEmail, "unknown_sender", "Sender not authorized for this thread")
}
if !participant.Verified {
return quarantine(ctx, rawEmail, "unverified_sender", "Sender not verified")
}
// Step 6: Create entry_event
event := &EntryEvent{
ID: uuid.New().String(),
EntryID: thread.EntryID,
ActorID: participant.UserID,
Channel: "email",
Action: "message",
Data: packEmailData(msg),
DKIMResult: dkimResult,
Timestamp: time.Now().UnixMilli(),
}
if err := createEntryEvent(ctx, event); err != nil {
return err
}
// Step 7: Update routing chain if applicable
return processRoutingTrigger(ctx, thread.EntryID, event)
}
```
### 5.4 Quarantine Flow
```sql
CREATE TABLE quarantine (
id TEXT PRIMARY KEY,
channel TEXT NOT NULL,
raw_data BLOB NOT NULL, -- packed: original message
reason TEXT NOT NULL,
details TEXT,
sender TEXT,
thread_id TEXT,
project_id TEXT, -- if determinable
reviewed_by TEXT REFERENCES users(id),
reviewed_at INTEGER,
disposition TEXT, -- "approved" | "rejected" | "spam"
created_at INTEGER NOT NULL
);
CREATE INDEX idx_quarantine_unreviewed ON quarantine(reviewed_at) WHERE reviewed_at IS NULL;
```
**Quarantine review workflow:**
1. IB admin sees quarantine queue
2. Reviews message content, sender, DKIM status
3. Options:
- **Approve:** Add sender to `channel_participants`, process message
- **Reject:** Discard, optionally notify sender
- **Spam:** Blacklist sender domain
---
## 6. Session Management
### 6.1 Session Model
```sql
CREATE TABLE sessions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
fingerprint TEXT NOT NULL, -- hashed device fingerprint
ip_address TEXT NOT NULL,
user_agent TEXT,
created_at INTEGER NOT NULL,
last_active_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
revoked_at INTEGER,
revoke_reason TEXT
);
CREATE INDEX idx_sessions_user ON sessions(user_id);
CREATE INDEX idx_sessions_active ON sessions(user_id, revoked_at) WHERE revoked_at IS NULL;
```
### 6.2 Single Active Session Enforcement
Users may have only ONE active session at a time (configurable per organization).
```go
func CreateSession(ctx context.Context, userID, fingerprint, ip, userAgent string) (*Session, error) {
// Check org session policy
org, _ := getUserOrg(ctx, userID)
if org.SingleSessionPolicy {
// Revoke all existing sessions for this user
if err := revokeUserSessions(ctx, userID, "new_session"); err != nil {
return nil, err
}
}
session := &Session{
ID: uuid.New().String(),
UserID: userID,
Fingerprint: hashFingerprint(fingerprint),
IPAddress: ip,
UserAgent: userAgent,
CreatedAt: time.Now().UnixMilli(),
LastActiveAt: time.Now().UnixMilli(),
ExpiresAt: time.Now().Add(7 * 24 * time.Hour).UnixMilli(),
}
return session, createSession(ctx, session)
}
```
### 6.3 Device Fingerprint Binding
Tokens are bound to device fingerprint. Fingerprint mismatch = token rejected.
```go
type Fingerprint struct {
UserAgent string
AcceptLanguage string
Timezone string
ScreenRes string // optional, JS-provided
ColorDepth int // optional
}
func HashFingerprint(fp *Fingerprint) string {
data, _ := json.Marshal(fp)
hash := sha256.Sum256(data)
return base64.RawURLEncoding.EncodeToString(hash[:16]) // 128-bit
}
func ValidateTokenFingerprint(token *JWT, currentFP *Fingerprint) error {
currentHash := HashFingerprint(currentFP)
if token.Claims.Fingerprint != currentHash {
return ErrFingerprintMismatch
}
return nil
}
```
### 6.4 Immediate Revocation on Access Change
When access is revoked, all sessions for that user in that project are invalidated.
```go
func RevokeAccess(ctx context.Context, accessID, revokedBy, reason string) error {
access, err := getAccess(ctx, accessID)
if err != nil {
return err
}
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
// Mark access as revoked
if err := markAccessRevoked(ctx, tx, accessID, revokedBy, time.Now().UnixMilli()); err != nil {
return err
}
// Revoke all user's tokens for this project
if err := revokeTokensForUserProject(ctx, tx, access.UserID, access.ProjectID, "access_revoked"); err != nil {
return err
}
// Audit log
if err := auditLog(ctx, tx, "access_revoked", revokedBy, access.UserID, map[string]interface{}{
"project_id": access.ProjectID,
"role": access.Role,
"reason": reason,
}); err != nil {
return err
}
return tx.Commit()
}
```
### 6.5 Session Timeout Behavior
| Activity State | Timeout |
|----------------|---------|
| Active (recent request) | Extend by access token lifetime (1h) |
| Idle | 15 minutes |
| Locked screen | Session paused, resume with re-auth |
| Browser closed | Session continues until refresh token expires |
---
## 7. Concurrency Control
### 7.1 Optimistic Locking with Version Field
All entries have a version field. Updates require matching version.
```sql
ALTER TABLE entries ADD COLUMN version INTEGER NOT NULL DEFAULT 1;
```
### 7.2 Update Pattern
```go
func EntryUpdate(ctx context.Context, entry *Entry, expectedVersion int) error {
result, err := db.ExecContext(ctx, `
UPDATE entries
SET summary = ?, data = ?, updated_at = ?, version = version + 1
WHERE entry_id = ? AND version = ?
`, entry.Summary, entry.Data, time.Now().UnixMilli(), entry.EntryID, expectedVersion)
if err != nil {
return err
}
rows, _ := result.RowsAffected()
if rows == 0 {
return ErrConcurrentModification
}
return nil
}
```
### 7.3 ETag Implementation
HTTP responses include ETag based on version + updated_at.
```go
func EntryETag(entry *Entry) string {
data := fmt.Sprintf("%s:%d:%d", entry.EntryID, entry.Version, entry.UpdatedAt)
hash := sha256.Sum256([]byte(data))
return fmt.Sprintf(`"%s"`, base64.RawURLEncoding.EncodeToString(hash[:8]))
}
```
**Request flow:**
1. GET returns `ETag: "abc123"` header
2. PUT/PATCH includes `If-Match: "abc123"` header
3. Server validates ETag matches current version
4. If mismatch: `412 Precondition Failed`
```go
func handleEntryUpdate(w http.ResponseWriter, r *http.Request) {
entryID := chi.URLParam(r, "id")
// Get current entry
entry, err := EntryRead(r.Context(), actorID, projectID, entryID)
if err != nil {
writeError(w, err)
return
}
// Validate ETag
ifMatch := r.Header.Get("If-Match")
currentETag := EntryETag(entry)
if ifMatch != "" && ifMatch != currentETag {
w.WriteHeader(http.StatusPreconditionFailed)
json.NewEncoder(w).Encode(map[string]interface{}{
"error": "concurrent_modification",
"current_etag": currentETag,
})
return
}
// Parse update
var update EntryUpdate
if err := json.NewDecoder(r.Body).Decode(&update); err != nil {
writeError(w, ErrBadRequest)
return
}
// Apply update with version check
if err := EntryUpdate(r.Context(), entry, entry.Version); err != nil {
if err == ErrConcurrentModification {
w.WriteHeader(http.StatusConflict)
json.NewEncoder(w).Encode(map[string]interface{}{
"error": "conflict",
"message": "Entry was modified by another user",
})
return
}
writeError(w, err)
return
}
// Return updated entry with new ETag
w.Header().Set("ETag", EntryETag(entry))
json.NewEncoder(w).Encode(entry)
}
```
### 7.4 Conflict Resolution UI Guidance
When conflict occurs, client should:
1. Fetch current version
2. Show diff between user's changes and current state
3. Allow user to merge or overwrite
4. Retry with new version
---
## 8. Rate Limiting
### 8.1 Limit Tiers
| Tier | Per-User | Per-IP | Per-Project | Notes |
|------|----------|--------|-------------|-------|
| Authentication | 5/min | 20/min | — | Prevents credential stuffing |
| API Read | 300/min | 1000/min | 5000/min | Standard operations |
| API Write | 60/min | 200/min | 1000/min | Mutations |
| File Upload | 10/min | 30/min | 100/min | Bandwidth protection |
| File Download | 50/min | 100/min | 500/min | Exfiltration protection |
| AI Matching | 20/min | 50/min | 200/min | Embedding cost control |
| Search | 30/min | 100/min | 500/min | Query cost control |
### 8.2 Implementation
Using sliding window counter with Redis:
```go
type RateLimiter struct {
redis *redis.Client
}
func (rl *RateLimiter) Allow(ctx context.Context, key string, limit int, window time.Duration) (bool, error) {
now := time.Now().UnixMilli()
windowStart := now - window.Milliseconds()
pipe := rl.redis.Pipeline()
// Remove old entries
pipe.ZRemRangeByScore(ctx, key, "0", strconv.FormatInt(windowStart, 10))
// Count current window
countCmd := pipe.ZCard(ctx, key)
// Add current request
pipe.ZAdd(ctx, key, redis.Z{Score: float64(now), Member: strconv.FormatInt(now, 10)})
// Set expiry
pipe.Expire(ctx, key, window)
if _, err := pipe.Exec(ctx); err != nil {
return false, err
}
count := countCmd.Val()
return count < int64(limit), nil
}
// Rate limit keys
func userRateLimitKey(userID, action string) string {
return fmt.Sprintf("rl:user:%s:%s", userID, action)
}
func ipRateLimitKey(ip, action string) string {
return fmt.Sprintf("rl:ip:%s:%s", ip, action)
}
func projectRateLimitKey(projectID, action string) string {
return fmt.Sprintf("rl:project:%s:%s", projectID, action)
}
```
### 8.3 Middleware Integration
```go
func RateLimitMiddleware(limiter *RateLimiter, action string, userLimit, ipLimit int, window time.Duration) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
userID := getUserID(ctx)
ip := getClientIP(r)
// Check user limit
if userID != "" {
allowed, err := limiter.Allow(ctx, userRateLimitKey(userID, action), userLimit, window)
if err != nil {
log.Error("rate limit check failed", "error", err)
} else if !allowed {
w.Header().Set("Retry-After", "60")
w.WriteHeader(http.StatusTooManyRequests)
json.NewEncoder(w).Encode(map[string]string{
"error": "rate_limit_exceeded",
"message": "Too many requests. Please wait before retrying.",
})
return
}
}
// Check IP limit
allowed, err := limiter.Allow(ctx, ipRateLimitKey(ip, action), ipLimit, window)
if err != nil {
log.Error("rate limit check failed", "error", err)
} else if !allowed {
w.Header().Set("Retry-After", "60")
w.WriteHeader(http.StatusTooManyRequests)
json.NewEncoder(w).Encode(map[string]string{
"error": "rate_limit_exceeded",
"message": "Too many requests from this IP.",
})
return
}
next.ServeHTTP(w, r)
})
}
}
```
### 8.4 Anomaly Detection Integration
Rate limiting feeds anomaly detection:
```go
type AnomalyEvent struct {
UserID string
IP string
Action string
Count int
Window time.Duration
Threshold int
Timestamp int64
}
func (rl *RateLimiter) CheckAnomaly(ctx context.Context, userID, ip, action string) {
// If user exceeds 80% of limit, log for analysis
// Pattern: bulk downloads, rapid enumeration, credential stuffing attempts
userKey := userRateLimitKey(userID, action)
count, _ := rl.redis.ZCard(ctx, userKey).Result()
// Get limit for this action
limit := getLimitForAction(action)
if float64(count) > float64(limit)*0.8 {
anomaly := &AnomalyEvent{
UserID: userID,
IP: ip,
Action: action,
Count: int(count),
Threshold: limit,
Timestamp: time.Now().UnixMilli(),
}
publishAnomaly(ctx, anomaly)
}
}
```
---
## 9. Audit Logging
### 9.1 Audit Events
All security-relevant events are logged immutably.
```sql
CREATE TABLE audit (
id TEXT PRIMARY KEY,
project_id TEXT,
actor_id TEXT NOT NULL,
action TEXT NOT NULL,
target_type TEXT, -- "entry" | "user" | "access" | "session" | "file"
target_id TEXT,
details TEXT NOT NULL, -- packed: JSON with action-specific data
ip_address TEXT NOT NULL,
user_agent TEXT,
timestamp INTEGER NOT NULL,
-- Immutability enforcement
hash TEXT NOT NULL, -- SHA-256 of (previous_hash + event_data)
previous_id TEXT -- chain reference
);
CREATE INDEX idx_audit_project ON audit(project_id, timestamp);
CREATE INDEX idx_audit_actor ON audit(actor_id, timestamp);
CREATE INDEX idx_audit_target ON audit(target_type, target_id, timestamp);
```
### 9.2 Event Types
| Action | Target | Details |
|--------|--------|---------|
| `auth.login` | user | mfa_used, ip, fingerprint |
| `auth.logout` | session | reason |
| `auth.mfa_enabled` | user | method |
| `auth.password_changed` | user | source (user, admin, reset) |
| `access.granted` | access | role, workstream, granted_by |
| `access.revoked` | access | reason, revoked_by |
| `entry.created` | entry | type, parent_id |
| `entry.updated` | entry | changed_fields |
| `entry.deleted` | entry | soft/hard |
| `entry.published` | entry | broadcast_to |
| `file.uploaded` | file | size, type, entry_id |
| `file.downloaded` | file | entry_id |
| `file.deleted` | file | reason |
| `session.created` | session | fingerprint, ip |
| `session.revoked` | session | reason |
| `key.rotated` | key_rotation | version, scope |
| `quarantine.reviewed` | quarantine | disposition |
### 9.3 Hash Chain Integrity
Each audit entry includes a hash of itself plus the previous entry, creating a tamper-evident chain.
```go
func CreateAuditEntry(ctx context.Context, entry *AuditEntry) error {
// Get previous entry hash
var previousHash string
var previousID string
prev, err := getLatestAuditEntry(ctx)
if err == nil && prev != nil {
previousHash = prev.Hash
previousID = prev.ID
}
// Calculate hash
data := fmt.Sprintf("%s:%s:%s:%s:%d:%s",
entry.ActorID, entry.Action, entry.TargetID,
entry.Details, entry.Timestamp, previousHash)
hash := sha256.Sum256([]byte(data))
entry.Hash = hex.EncodeToString(hash[:])
entry.PreviousID = previousID
return insertAuditEntry(ctx, entry)
}
func VerifyAuditChain(ctx context.Context, from, to int64) error {
entries, err := getAuditEntriesInRange(ctx, from, to)
if err != nil {
return err
}
var previousHash string
for _, entry := range entries {
data := fmt.Sprintf("%s:%s:%s:%s:%d:%s",
entry.ActorID, entry.Action, entry.TargetID,
entry.Details, entry.Timestamp, previousHash)
expectedHash := sha256.Sum256([]byte(data))
if entry.Hash != hex.EncodeToString(expectedHash[:]) {
return fmt.Errorf("audit chain broken at entry %s", entry.ID)
}
previousHash = entry.Hash
}
return nil
}
```
---
## 10. Security Checklist for Implementation
### 10.1 Pre-Launch
- [ ] BoringCrypto build verified (`crypto/boring.Enabled()` returns true)
- [ ] Master key stored in HSM/Vault with Shamir split
- [ ] All JWT signing keys derived via HKDF-SHA256
- [ ] Blind indexes implemented (no deterministic encryption)
- [ ] MFA mandatory for all ib_admin and ib_member roles
- [ ] DKIM verification enabled for email ingestion
- [ ] Rate limiting active on all endpoints
- [ ] Audit logging capturing all security events
- [ ] Session single-login policy configurable per org
- [ ] ETags implemented on all entry mutations
### 10.2 Ongoing
- [ ] Key rotation drill performed quarterly
- [ ] Audit chain integrity verified weekly
- [ ] Rate limit thresholds reviewed monthly
- [ ] Quarantine queue reviewed daily
- [ ] Session anomaly reports reviewed weekly
- [ ] Penetration test annually
- [ ] SOC 2 Type II audit annually
---
*This specification is the authoritative source for Dealspace security architecture. If implementation diverges from this spec, the implementation is wrong.*
Reference in New Issue View Git Blame Copy Permalink