avi/S.H.O.N.A.R.
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

P3-M7: providers, background sync, AI pipeline

Browse source 
P3 CustomShonarProvider (auth, chunked uploads, secure token store).
P4 NextcloudProvider (login flow v2, DAV, chunking) + FolderSyncProvider
(Syncthing-style). P5 TOFU pinning + redacting logger + leak tests.
P6a generic hosted setup (Start9/Umbrel URL + auto-detect). P7 provider
switching (Room v2 slot, Storage screen, foreground migrator). M5
WorkManager drain (Room v3 backoff). M7 AI pipeline (4 adapters, arq
worker, transcript/summary/jobs endpoints). Docs updated throughout.
This commit is contained in:
avi 2026-09-12 11:29:55 -05:00
commit aad7cad78d
73 changed files with 9470 additions and 54 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/ROADMAP.md
View file

@ -9,13 +9,13 @@ updated in the same commit as the work it describes.
| M0 | Repo scaffold, license, docs, Docker dev stack, `/healthz` `/readyz`, Alembic schema | done |
| M1 | Auth: register / login / rotating refresh + reuse detection / logout / delete-account, Argon2id, rate limits | done |
| M2 | Upload sessions (chunked, resumable), storage abstraction (local + S3), recordings CRUD, ownership checks | done |
| M3 | Android: server URL config, login, token persistence + auto-refresh | TODO |
| M3 | Android: server URL config, login, token persistence + auto-refresh | implemented via P3 `CustomShonarProvider` (Keystore token store, transparent refresh, login UI in provider selection); on-device verification pending |
| S-1 | Android: generic Custom Settings engine (8 types, validation, custom CRUD, import/export, secure storage) — 19 unit tests green | done |
| ~~HA-1/HA-2~~ | Home Assistant integration (client, repository, devices screen, e2e scripts) | **DEFERRED — out of initial product scope; preserved under `deferred/home-assistant/` and branch `deferred/home-assistant`** |
| M4 | Android: foreground-service recording (pause/resume/stop), metadata, Room | TODO |
| M5 | Android: WorkManager upload sync (retry, Wi-Fi-only, charging-only, pause) | TODO |
| M6 | Android: library (search/filter/sort), playback (seek/speed), waveform, download/delete | TODO |
| M7 | Backend: AI pipeline + adapters (whisper_http, faster-whisper, OpenAI-compat, Ollama, none), status endpoints | TODO |
| M4 | Android: foreground-service recording (pause/resume/stop), metadata, Room | implemented; on-device verification pending |
| M5 | Android: WorkManager upload sync (retry, Wi-Fi-only, charging-only, pause) | done — `SyncWorker` + `SyncScheduler` (15-min periodic, one-shot on startup/constraint change, backoff on retry), Context-free `SyncDrain` engine (drains QUEUED + due ERROR incl. P7 leftovers, exponential backoff 1m–1h, cancel reverts to QUEUED), pause as resting state; full suite 167 green. On-device behaviour pending |
| M6 | Android: library (search/filter/sort), playback (seek/speed), waveform, download/delete | partial; local list, playback, and delete implemented |
| M7 | Backend: AI pipeline + adapters (whisper_http, faster-whisper, OpenAI-compat, Ollama, none), status endpoints | done — provider protocols + 4 adapters (faster-whisper lazy optional), versioned transcripts/summaries (user edits win), arq worker (`run_transcribe`/`run_summarize` + startup/5-min sweep, max 3 tries), transcript/summary/jobs endpoints, `none` means skipped; 50 backend tests green, ruff clean |
| M8 | Android: details screen — transcript synced to playback, summary, action items, editing | TODO |
| M9 | Backend: full-text search endpoints + filters, exports (audio/txt/md/zip), deletion sweep | TODO (schema/FTS columns exist) |
| M10 | Android dark mode, accessibility pass, consent UX polish, deploy/backup docs, OpenAPI sync | TODO |

16
docs/server-providers.md
View file

@ -142,6 +142,11 @@ provider API first, local wipe second.
## 7. Start9 / Umbrel / custom adapters
> Status: P6a shipped the generic path — Start9/Umbrel cards take a service
> URL and auto-detect Nextcloud vs SHONAR, handing off to those providers.
> The platform-RPC discovery below, multi-service picking, and Tor onion
> access (Orbot SOCKS) are deferred to P6b.
- **Start9 & Umbrel are platforms, not APIs.** The adapter pattern is
"platform probe + service binding": a `PlatformProbe` (Start9: Server API
over its RPC; Umbrel: its app manifest endpoints, where available)
@ -182,11 +187,12 @@ provider API first, local wipe second.
| P0 (done) | HA isolated to `deferred/` + branch; removed from build, onboarding, defaults, tests | build + unit tests green |
| P1 | `provider/` module: `ShonarProvider` interface, `ServerUrl` validation, `LocalOnlyProvider`, `ProviderRegistry` | contract suite (local-only) |
| P2 | Provider-selection onboarding screen + StorageLocation screen wired to registry | on-device |
| P3 | `CustomShonarProvider` against this repo's backend (auth M1 + uploads M2), WorkManager sync states | backend + app integration |
| P4 | `NextcloudProvider`: login flow v2, DAV upload/download/delete, chunking | contract suite + a real Nextcloud instance |
| P5 | TLS TOFU pinning + redaction/logging + leak tests | cert fixtures |
| P6 | Platform probes for Start9/Umbrel + service-binding UX | probe fakes |
| P7 | Provider switching w/ migration prompts, account deletion, revocation | e2e |
| P3 (done) | `CustomShonarProvider` against this repo's backend (auth M1 + uploads M2), WorkManager sync states | contract suite (10 shared + 12 custom) + 9 auth tests vs MockWebServer fixtures mirroring `schemas_*.py`; backend pytest 28 green on live Postgres; `assembleDebug` clean |
| P4 (done) | `NextcloudProvider`: login flow v2, DAV upload/download/delete, chunking — plus `FolderSyncProvider` (sync-folder for Syncthing etc., requested alongside) | contract suite (10 shared + 6 custom) + 6 auth + 4 protocol tests vs MockWebServer fake; DAV/chunking-v2 protocol verified live against Nextcloud 34 (MKCOL/PUT/MOVE-assemble/PROPFIND/OCS/login-v2 via curl); full suite 120 green, `assembleDebug` clean. On-device browser-approval tap-through still pending |
| P5 (done) | TLS TOFU pinning + redaction/logging + leak tests | TOFU trust manager (system-first, per-host DER pins in secure store, explicit approval UI with fingerprint + issuer + validity) + redacting logger gated by `log_http_bodies` (transcript/sidecar bodies never log, in any mode) + HeldCertificate TLS fixtures; full suite 138 green, `assembleDebug` clean. Self-signed hosts now connect after one approval; hostname verification stays strict |
| P6a (done) | Generic hosted setup: Start9/Umbrel cards take a service URL, auto-detect Nextcloud vs SHONAR, hand off to the existing provider flows; the platform is an entry path, the persisted provider is the protocol | routing matrix unit tests (`routeHosted`); full suite + `assembleDebug` |
| P6b (deferred) | Platform RPC auto-discovery (Start9 Server API, Umbrel manifests), multi-service picker, Tor onion access via Orbot SOCKS | probe fakes + on-device, when needed |
| P7 (done) | Provider switching w/ migration prompts, account deletion, revocation | Room v2 provider slot per recording (origin + remote ref + sync state, honest migration) + "Where is my data?" screen (summary, reconnect, disconnect+revoke, delete account) + foreground migrate-now uploader (bounded: cancellable, one attempt/file, upload-only) + switch-time prompt (upload / keep / forget links); migration matrix + slots + converter tests, full suite 156 green, `assembleDebug` clean. Background/scheduled sync stays M5; pull-down and old-remote wipe are explicit non-goals (a future wipe is safe: everything lives under `SHONAR/` prefixes) |
Recording engine (M4) and playback (M6) proceed independently on top of the
same Room model; provider work is orthogonal.