Lumen/SPIKE-02-ATOMIC-DATASET-UPDATES.md

SPIKE-02 — Atomic Dataset Updates (A/B Slots)

• Phase: Architecture Validation
• Date: 2026-08-30
• Decision under test: ADR-006 (A/B dual-slot, staged download, atomic activation, rollback).
• Method: adversarial state-machine simulation with crash/fault injection at every stage (experiments/exp2-ab-update-sim.mjs), plus analysis of the ordering that makes single-database atomicity sufficient.

1. The invariant under test

"Either the previous valid dataset remains active, or the new valid dataset becomes active. The application never knowingly exposes a partial dataset."

Four distinct mechanisms must cooperate for this to hold; they are often confused, so they are separated here explicitly:

Mechanism	What it guarantees	What it does NOT guarantee
Atomic database operation (single IDB transaction)	The pointer flip and its metadata land together or not at all.	That the dataset being pointed at is complete or valid.
Validation (signature → compatibility → hashes → schema)	That a dataset is eligible to become active.	That it will be activated, or survive after activation.
Activation (commit pointer to a validated dataset)	That the active slot is, at the moment of commit, a validated dataset.	That it stays uncorrupted afterwards.
Crash recovery (boot light-verify + fallback + readback)	That a previously-committed good state is found and served after any interruption.	Prevention of the interruption itself.

The invariant is a property of the composition of all four, not of any one.

2. Verification ordering (normative)

Cheapest and most authoritative checks first, so untrusted data never forces work:

1. Fetch latest.json → monotonic version comparison (reject ≤ active).
2. Fetch candidate manifest.json.
3. Signature over manifest digest (authenticity) — before parsing or downloading anything else.
4. Parse manifest → compatibility (appCompatibility, schemaVersion in shell range) and size budgets — before downloading files.
5. Stage files into the inactive slot; per-file SHA-256 + size check on each (integrity), one short transaction per file.
6. Schema validation of staged sections.
7. Activate: single transaction on lumen-system flipping pointer + version + verification record.
8. Readback spot-check after activation (and at next boot if pending).

3. Fourteen-step walkthrough (results from EXP-2)

exp2-ab-update-sim.mjs models slots as atomic stores and injects faults at each stage. Boot runs light verification and falls back to the other slot when the active slot fails. 16/16 scenarios PASS.

#	Step / injected fault	Active dataset after	Result
1	Download begins, crash before any file	Old (v1)	PASS
2	Download partially completes (3/5 files)	Old (v1)	PASS
3	Browser terminated mid-staging	Old (v1)	PASS
4	Phone reboots before activation	Old (v1)	PASS
5	Dataset validation fails (generic)	Old (v1), candidate rejected	PASS
6	Integrity hash fails (corrupt file)	Old (v1), candidate rejected	PASS
7	Signature validation fails	Old (v1), nothing downloaded	PASS
8	Schema validation fails	Old (v1), candidate rejected	PASS
9	Compatibility validation fails	Old (v1), nothing downloaded	PASS
10	Activation succeeds	New (v2)	PASS
11	Pointer update occurs	(covered by 10/12)	PASS
12	Browser terminates immediately after flip	New (v2) — flip committed; boot confirms	PASS
13	App starts again (crash during flip)	Old (v1) — transaction never committed	PASS
14	Recovery: corruption found by readback	Rollback to last complete slot	PASS

Extra scenarios: active slot corrupted at boot → fallback slot served (PASS); both slots lost → RECOVERY state with embedded emergency baseline and favorites intact (PASS); favorites survive a full update (PASS).

4. Why single-database atomicity is sufficient

There is no cross-database transaction in IDB, and the pointer lives in a different database than the dataset files. This is safe because of ordering, not because of a distributed transaction:

• The inactive slot is fully written and validated before the pointer can move. The flip therefore only ever points at an already-complete dataset.
• The flip itself is one transaction in one database (lumen-system), so the pointer, version, and verification record cannot disagree with each other.
• If the process dies at any point: before flip → old pointer stands; during flip → transaction uncommitted, old pointer stands; after flip → new pointer stands and boot light-verify confirms the slot it points at.
• Post-activation corruption (rare) is caught by readback/next-boot light verification, which falls back to the other slot if complete, else RECOVERY.

5. Findings and refinements

• F-1 (add): per-file staging must commit bytes + progress record in the same transaction. A separate progress write could orphan a record after a crash. (Carried to ADR-006 addendum; simulation already models this.)
• F-2 (add): the activation transaction must include the verification record (what was verified, when, which version), so boot can trust the active slot without re-hashing everything.
• F-3 (add): a readbackPending flag is set in the activation transaction and cleared after a successful readback (same boot if immediate, next boot otherwise). Scenario 12's crash window is thereby explicitly covered.
• F-4 (accepted trade-off, document): rollback depth is exactly one. Beginning a new staging run reuses (wipes) the inactive slot, which is where the previous rollback copy lived. Invariant still holds (a valid dataset is always active); only the depth of undo is limited. Three-slot storage was considered and rejected for footprint.
• F-5 (confirmed): version monotonicity + quarantine of rejected versions prevents replay of a known-bad package.
• F-6 (confirmed): eviction is per-origin all-or-nothing, so there is no "one database evicted, another survives" partial state to handle; eviction routes to RECOVERY + baseline (SPIKE-01 §3.9).

6. What this spike did NOT prove

• It simulated the state machine, not IndexedDB's physical transaction semantics. The claim "an IDB transaction is atomic on iOS under jetsam kill" is INFERRED from spec + platform documentation and remains UNVERIFIED on physical iOS (SPIKE-01 §5).
• It did not measure staging throughput (device-dependent).

7. Verdict

ACCEPT WITH CHANGES.

The A/B architecture genuinely guarantees the invariant given IDB transaction atomicity holds on target devices (the one platform assumption, isolated and queued for physical testing in SPIKE-01 §5). Changes adopted: F-1, F-2, F-3 added to ADR-006; F-4 documented as an accepted limitation.