Stage 1: project foundation (strict TS, lint boundaries B-1..B-7, directory structure, CI, boundary tests)
- dedicated git repo at /home/avi/Projects/Lumen (main)
- TypeScript strict (target ES2022, bundler, exactOptionalPropertyTypes, noUncheckedIndexedAccess)
- ESLint 9 + typescript-eslint strictTypeChecked + eslint-plugin-boundaries for B-1..B-7, no-restricted-globals/syntax for B-1/B-7
- Prettier 3.5
- Structure per IMPLEMENTATION-CONTRACT.md §4 (src/platform/idb|cache|sw, storage, data, sync/{transport,verifier}, domain/{emergency,schedule,map,festival,readiness,clock,favorites}, ui/{components,views,router,render}, app, emergency-baseline, assets, public, content, pipeline, tests, scripts)
- CI: .github/workflows/ci.yml (typecheck + lint + format + test)
- Boundary tests: tests/unit/boundaries.test.ts (4 tests) + scripts/check-boundaries.ts
- No feature code, no PWA/IDB/sync/mesh/accounts per contract Stage 1
This commit is contained in:
Lumen Stage1
commit
c0bfd413ff
100 changed files with 9863 additions and 0 deletions
191
SPIKE-01-INDEXEDDB-IOS.md
Normal file
191
SPIKE-01-INDEXEDDB-IOS.md
Normal file
|
|
@ -0,0 +1,191 @@
|
|||
# SPIKE-01 — IndexedDB + iOS Reliability
|
||||
|
||||
- **Phase:** Architecture Validation
|
||||
- **Date:** 2026-08-30
|
||||
- **Decision under test:** ADR-004 (IndexedDB as primary local storage; A/B
|
||||
slot layout) and the storage foundations of ADR-006.
|
||||
- **Environment limitation (declared up front):** No iOS hardware or iOS
|
||||
Simulator is reachable from this Linux dev box. Nothing iOS-specific below
|
||||
was observed here. iOS items are labelled **INFERRED** (from WebKit's own
|
||||
published storage policy) or **UNVERIFIED** (needs a physical iPhone). This
|
||||
spike does **not** pretend otherwise.
|
||||
|
||||
## 1. Question
|
||||
|
||||
Is IndexedDB, organized as A/B dataset slots + a system-meta store + a
|
||||
user-data store, a sound foundation for Lumen on iOS Safari and Android
|
||||
Chrome, across large datasets, large map assets, crashes, restarts, storage
|
||||
pressure, and eviction?
|
||||
|
||||
## 2. Workload being validated
|
||||
|
||||
| Dimension | Lumen value (budget, ARCH §10.6) |
|
||||
|---|---|
|
||||
| Structured section JSON | ≤ 3 MB total |
|
||||
| Map assets (WebP) | ≤ 28 MB |
|
||||
| Other assets | ≤ 6 MB |
|
||||
| Total dataset | ≤ 40 MB target / 50 MB ceiling |
|
||||
| Worst case on device (A/B + shell + user) | ≈ 2×40 MB + 1 MB + ε < 90 MB |
|
||||
| Object stores | `files`, `assets` per slot DB; `meta` in system DB; `favorites`, `prefs`, `diag` in user DB |
|
||||
| Largest single record | ≤ 6 MB (per-file cap, ARCH §10.6) |
|
||||
|
||||
## 3. Findings, item by item
|
||||
|
||||
Each item is labelled. Citations are to vendor/spec documentation current as of
|
||||
2026 (see DISCOVERY Appendix B for the underlying sources).
|
||||
|
||||
### 3.1 Large festival datasets (tens of MB)
|
||||
- **INFERRED.** IndexedDB is explicitly the browser's intended store for large
|
||||
structured data; quotas on both targets are expressed as a fraction of disk
|
||||
(WebKit: up to ~60% of disk per origin for browser apps since Safari 17;
|
||||
Chromium similar). A 40 MB dataset is a tiny fraction of any realistic quota.
|
||||
- Risk is not quota *size*; it is *eviction policy* (see 3.9) and *write
|
||||
interruption* (3.10).
|
||||
|
||||
### 3.2 Multiple object stores / multiple databases
|
||||
- **CONFIRMED (spec):** multiple databases and multiple object stores per
|
||||
origin are standard; transactions are scoped to one database.
|
||||
- **INFERRED:** our layout (two slot DBs + system DB + user DB) is well within
|
||||
normal use. Note: there is **no cross-database transaction** — the A/B design
|
||||
already accounts for this (activation is a single-DB transaction on the
|
||||
system DB; see SPIKE-02).
|
||||
|
||||
### 3.3 Large map assets (multi-MB Blobs in IDB)
|
||||
- **INFERRED.** IDB stores Blobs; multi-MB records are supported. We cap single
|
||||
records at 6 MB to bound transaction duration and memory.
|
||||
- **UNVERIFIED (both platforms):** sustained read-back performance of ~28 MB of
|
||||
WebP blobs into object URLs, and behavior when writing them near quota. Must
|
||||
be measured on real devices (Android low-end and iPhone).
|
||||
|
||||
### 3.4 Transaction behavior
|
||||
- **CONFIRMED (spec):** IDB transactions are atomic per database; a transaction
|
||||
either fully applies or does not. `oncomplete`/`onerror`/`onabort` signal the
|
||||
outcome. Versionchange transactions can block reads/writes.
|
||||
- **INFERRED:** keeping transactions short (one file + its progress record per
|
||||
transaction) avoids pathological lock holding and keeps the A/B flip cheap.
|
||||
|
||||
### 3.5 Atomicity
|
||||
- **CONFIRMED (spec):** within one database, a transaction is all-or-nothing.
|
||||
This is exactly what ADR-006's activation flip relies on (single transaction
|
||||
on `lumen-system`).
|
||||
- **CONFIRMED (spec):** atomicity does **not** span databases. The architecture
|
||||
must not assume it does (SPIKE-02 validates the ordering that makes this safe).
|
||||
|
||||
### 3.6 Browser termination (tab/process killed)
|
||||
- **INFERRED.** If the process dies mid-transaction, the transaction does not
|
||||
commit; durable state reverts to the last committed transaction. This is the
|
||||
standard crash-consistency model for IDB and is what EXP-2's crash points
|
||||
simulate.
|
||||
- **UNVERIFIED (iOS):** exact behavior when Safari force-quits or iOS jetsam
|
||||
kills the PWA process mid-write is expected to follow the same model but must
|
||||
be observed on device.
|
||||
|
||||
### 3.7 Service worker termination
|
||||
- **CONFIRMED (architecture):** dataset staging runs in the **page** context,
|
||||
not the SW (constraint C-21), so SW termination cannot interrupt dataset
|
||||
writes. The SW only handles shell caching.
|
||||
- **INFERRED:** iOS aggressively terminates idle SWs; because our design keeps
|
||||
no dataset state in the SW, this is a non-issue for data integrity.
|
||||
|
||||
### 3.8 Browser restart / phone restart
|
||||
- **INFERRED.** Committed IDB data survives browser and device restart; the SW
|
||||
and caches re-register/rehydrate on next launch. Boot-time light verification
|
||||
(SPIKE-05) is the mechanism that confirms this at runtime.
|
||||
- **UNVERIFIED (iOS):** post-restart cold-start latency and SW re-registration
|
||||
timing on real iPhones.
|
||||
|
||||
### 3.9 Storage pressure & eviction
|
||||
- **CONFIRMED (WebKit policy):** storage is **best-effort**. Under pressure,
|
||||
WebKit evicts on an LRU basis across origins; `navigator.storage.persist()`
|
||||
is a *request* granted by heuristic (home-screen install helps), not a
|
||||
guarantee. Eviction is **per-origin and all-or-nothing** (IDB + Cache + SW
|
||||
registration together).
|
||||
- **CONFIRMED (Chromium policy):** similar best-effort LRU eviction; installed
|
||||
PWAs with persistence granted are skipped first.
|
||||
- **INFERRED:** an installed PWA with `persist()` granted and regular use is
|
||||
unlikely to be evicted during a festival weekend, but **nothing guarantees
|
||||
it**. Therefore eviction is treated as an expected failure mode with a
|
||||
recovery path (RECOVERY state + embedded emergency baseline), never as an
|
||||
impossible one.
|
||||
|
||||
### 3.10 Interrupted writes
|
||||
- **CONFIRMED (spec):** an interrupted/uncommitted write leaves prior committed
|
||||
state intact. Combined with per-file staging transactions (SPIKE-02), an
|
||||
interrupted download can leave a slot *partially staged* but can **never**
|
||||
corrupt the active dataset.
|
||||
- **INFERRED:** resume logic keyed on committed per-file progress records
|
||||
recovers cleanly; EXP-2 exercises this.
|
||||
|
||||
### 3.11 Read consistency
|
||||
- **CONFIRMED (spec):** reads inside a transaction see a consistent snapshot;
|
||||
reads outside a transaction see the latest committed state. Our read paths
|
||||
(DatasetStore) read committed state; there are no mid-update reads of the
|
||||
inactive slot by features.
|
||||
|
||||
### 3.12 iOS Safari specifics
|
||||
- **INFERRED (WebKit documented):** 7-day ITP cap applies to *Safari-tab* usage
|
||||
and is **exempt for home-screen-installed** PWAs; Safari 17+ quota is a disk
|
||||
fraction; eviction all-or-nothing.
|
||||
- **UNVERIFIED:** all of the above *as experienced by our specific app* — install
|
||||
exemption in practice, `persist()` grant rate, near-quota write behavior, Blob
|
||||
read-back speed, post-restart rehydration. Requires physical iOS.
|
||||
|
||||
### 3.13 Android Chrome specifics
|
||||
- **INFERRED (Chromium documented):** generous disk-fraction quotas; LRU
|
||||
eviction under pressure; installed PWAs + `persist()` favored.
|
||||
- **UNVERIFIED:** low-end-device IDB write throughput and Blob handling for a
|
||||
40 MB dataset; behavior under actual storage pressure on a 32–64 GB device
|
||||
nearly full.
|
||||
|
||||
## 4. Defensive protocol derived from this spike
|
||||
|
||||
These rules are the "changes" attached to ACCEPT-WITH-CHANGES and are carried
|
||||
into the ADR-004 addendum:
|
||||
|
||||
- **P1:** One short transaction per staged file (bytes + progress record commit
|
||||
together). No transaction spans an `await` of non-IDB work.
|
||||
- **P2:** Activation is exactly one transaction on the system DB.
|
||||
- **P3:** Every IDB call is wrapped; `QuotaExceededError` aborts staging and
|
||||
keeps the active dataset (never a crash).
|
||||
- **P4:** Pre-stage free-space check via `navigator.storage.estimate()`; refuse
|
||||
to stage if free < 2× package size.
|
||||
- **P5:** Single record ≤ 6 MB (bounds transaction memory/duration).
|
||||
- **P6:** Request `navigator.storage.persist()` once after READY; never rely on
|
||||
the grant.
|
||||
- **P7:** Boot-time light verification assumes storage may have vanished
|
||||
(eviction) — detection, not prevention.
|
||||
- **P8:** No dataset state in the Service Worker.
|
||||
|
||||
## 5. Must test on physical devices before production
|
||||
|
||||
### iOS (iPhone, iOS 16.4 low bound + latest)
|
||||
1. Fresh install → full prep of a real-size (~40 MB) package → force-quit and
|
||||
reboot → data intact, boot ≤ budget.
|
||||
2. Leave unused in Safari *tab* for > 7 days vs installed PWA → confirm tab
|
||||
data evicted, installed data survives.
|
||||
3. `navigator.storage.persist()` grant outcome when installed.
|
||||
4. Write dataset while device storage is nearly full → observe
|
||||
QuotaExceededError handling (no crash, active preserved).
|
||||
5. Kill app mid-download repeatedly → resume completes; active never partial.
|
||||
6. Blob read-back: open map with ~28 MB assets → time-to-interactive.
|
||||
|
||||
### Android (2021 mid-range, Chrome ~110 + latest)
|
||||
7. Same fresh-install/reboot cycle with 40 MB package.
|
||||
8. Storage-pressure eviction simulation (devtools / fill disk) → RECOVERY path.
|
||||
9. IDB write throughput for 40 MB staging; per-file transaction timing.
|
||||
10. Map asset Blob read-back performance on low-end GPU/RAM.
|
||||
|
||||
## 6. Verdict
|
||||
|
||||
**ACCEPT WITH CHANGES.**
|
||||
|
||||
- IndexedDB is the correct primary store (structure, transactions, Blob
|
||||
support, quota scale all fit). No alternative (OPFS, SQLite-WASM,
|
||||
localStorage) is better for this workload (see ADR-004).
|
||||
- The A/B layout is sound and its atomicity claims are validated at the
|
||||
design-logic level by EXP-2 (SPIKE-02).
|
||||
- **Changes:** adopt defensive protocol P1–P8 (added to ADR-004).
|
||||
- **Condition:** the physical-device tests in §5 are mandatory before the
|
||||
storage layer is declared production-ready. Until then the decision stands as
|
||||
the best-supported choice, with the platform-specific behaviors above
|
||||
explicitly **UNVERIFIED**, not confirmed.
|
||||
Loading…
Add table
Add a link
Reference in a new issue