Local-only app: remove network, uploads, providers, backend, desktop
Phone is now a pure on-device recorder: no accounts, no servers, no background uploads (INTERNET permission gone). File sync is the user's own tooling; the library adopts externally added files. Android: - Delete provider package (Nextcloud, custom SHONAR, sync-folder, registry, auth, TOFU/TLS), sync stack (SyncWorker/Drain/Slots, MigrationRunner), provider/storage/folder UI, AI-via-server details. - Home shows a fixed 'on this phone' library; details keep playback, rename, file info. Settings lose Network/provider/HTTP-logging. - Library root is the stored folder or Music/Recordings; import is double-scan proof (mutex + unique filePath index, migration v3->v4 dedupes by path) with regression tests. - Drop okhttp/work/security-crypto/media deps; delete their tests. Repo: backend/, desktop/, worker/, deploy/, shared/, server scripts and docs removed; README rewritten; CI keeps the android job only.
This commit is contained in:
avi
parent
7b3436837a
commit
8c510f4ab9
154 changed files with 196 additions and 23570 deletions
|
|
@ -1,36 +0,0 @@
|
|||
# Roadmap & feature status
|
||||
|
||||
Every checked item is implemented, tested, and present at the current HEAD.
|
||||
Anything not listed is aspirational. This table is maintained by hand and
|
||||
updated in the same commit as the work it describes.
|
||||
|
||||
| Milestone | Scope | Status |
|
||||
|---|---|---|
|
||||
| M0 | Repo scaffold, license, docs, Docker dev stack, `/healthz` `/readyz`, Alembic schema | done |
|
||||
| M1 | Auth: register / login / rotating refresh + reuse detection / logout / delete-account, Argon2id, rate limits | done |
|
||||
| M2 | Upload sessions (chunked, resumable), storage abstraction (local + S3), recordings CRUD, ownership checks | done |
|
||||
| M3 | Android: server URL config, login, token persistence + auto-refresh | implemented via P3 `CustomShonarProvider` (Keystore token store, transparent refresh, login UI in provider selection); on-device verification pending |
|
||||
| S-1 | Android: generic Custom Settings engine (8 types, validation, custom CRUD, import/export, secure storage) — 19 unit tests green | done |
|
||||
| ~~HA-1/HA-2~~ | Home Assistant integration (client, repository, devices screen, e2e scripts) | **DEFERRED — out of initial product scope; preserved under `deferred/home-assistant/` and branch `deferred/home-assistant`** |
|
||||
| M4 | Android: foreground-service recording (pause/resume/stop), metadata, Room | implemented; on-device verification pending |
|
||||
| M5 | Android: WorkManager upload sync (retry, Wi-Fi-only, charging-only, pause) | done — `SyncWorker` + `SyncScheduler` (15-min periodic, one-shot on startup/constraint change, backoff on retry), Context-free `SyncDrain` engine (drains QUEUED + due ERROR incl. P7 leftovers, exponential backoff 1m–1h, cancel reverts to QUEUED), pause as resting state; full suite 167 green. On-device behaviour pending |
|
||||
| M6 | Android: library (search/filter/sort), playback (seek/speed), waveform, download/delete | partial; local list, playback, and delete implemented |
|
||||
| M7 | Backend: AI pipeline + adapters (whisper_http, faster-whisper, OpenAI-compat, Ollama, none), status endpoints | done — provider protocols + 4 adapters (faster-whisper lazy optional), versioned transcripts/summaries (user edits win), arq worker (`run_transcribe`/`run_summarize` + startup/5-min sweep, max 3 tries), transcript/summary/jobs endpoints, `none` means skipped; 50 backend tests green, ruff clean |
|
||||
| M8 | Android: details screen — transcript synced to playback, summary, action items, editing | done — `detail/{id}` route (transcript/summary/status tabs, tap-to-seek, speed control, edit dialogs, title rename) on `CustomShonarProvider` AI methods (fetch/update transcript+summary, jobs, PATCH title/notes) against new backend `PUT transcript/summary` user-edit endpoints (versioned, pipeline won't clobber); pure `AiContent` parsing/sync mapping; 19 new Android tests + 5 backend edit tests, full suites green (186 Android, 55 backend), `assembleDebug` clean. Local-only/unsynced rows get honest empty states; on-device verification pending |
|
||||
| T1 | Backend transcription model support — registry (tiny/base/small/medium/large-v3), global default (`base`, runtime-editable), per-recording overrides, `GET /api/v1/models`, worker uses saved model, job stage/progress | done — migration (`recordings`+`upload_sessions.transcription_model`, `processing_jobs.stage/progress`, `app_settings`); `PUT /models/default` (future rows only); finalize/session override (finalize wins); faster-whisper model cache + fail-fast unavailable errors; 10 new tests, backend suite 64 green + ruff clean |
|
||||
| M9 | Backend: full-text search endpoints + filters, exports (audio/txt/md/zip), deletion sweep | done — `GET /search` (Postgres tsvector via migration fts0000000001, SQLite LIKE fallback for the desktop engine; scopes title/notes/transcript/summary/tag, ranked + snippets); list filters on `GET /recordings` (tag/status/from/to); `GET /recordings/{id}/export?fmt=audio|txt|md|zip` (inline artifacts, ExportJob+Asset audit trail); retention sweep (`services/retention.py`, 30-day grace `SHONAR_RETENTION_GRACE_DAYS`, wired into arq cron daily + inline-queue timer); 9 new tests, backend suite 78 green + ruff clean |
|
||||
| D-1 | Desktop: in-place reprocess (no re-upload) — `<name>.shonar.json` sidecar maps library file → server recording id; Re-transcribe/Re-summarize hit `POST /reprocess` (model override switches persist server-side); rename carries mapping; deleted-remote mapping self-heals to re-upload | done — provider `reprocess()`, endpoint `?model=` param, 2 backend + 6 desktop tests, suites green (69 backend, 23 desktop) |
|
||||
| M10 | Android dark mode, accessibility pass, consent UX polish, deploy/backup docs, OpenAPI sync | TODO |
|
||||
|
||||
## Explicit TODOs (not yet implemented)
|
||||
|
||||
- **Speaker diarization**: adapter interface planned; next step is a
|
||||
`DiarizationProvider` protocol + pyannote-audio adapter behind
|
||||
`SHONAR_DIARIZATION_PROVIDER`.
|
||||
- **At-rest encryption**: design in `docs/security.md`; implementation not
|
||||
started. Files are stored unencrypted unless you encrypt the volume.
|
||||
- **Meilisearch/OpenSearch search backend**: only Postgres FTS is planned to
|
||||
ship first, behind a `SearchBackend` protocol.
|
||||
- ~~**Account purge sweep**~~: done in M9 — `services/retention.py`
|
||||
hard-deletes expired soft-deletes (recordings + accounts + files);
|
||||
runs daily via arq cron and the desktop inline-queue timer.
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
# Architecture
|
||||
|
||||
```
|
||||
Android app Your server
|
||||
┌───────────────────┐ ┌──────────────────────────────────┐
|
||||
│ Compose UI │ │ nginx (TLS) │
|
||||
│ ViewModels │ HTTPS│ └─ FastAPI app ──► PostgreSQL │
|
||||
│ Room (offline DB) │◄─────┤ │ (records, │
|
||||
│ WorkManager │ │ │ FTS) │
|
||||
│ └ upload workers │ │ Redis◄┘ │
|
||||
│ MediaRecorder │ │ └─ arq worker ──► storage/ │
|
||||
│ Foreground service│ │ │ (originals + │
|
||||
│ Media3 player │ │ │ derivatives) │
|
||||
└───────────────────┘ │ AI adapters (optional): │
|
||||
│ STT: whisper_http|faster-whisper│
|
||||
│ LLM: ollama|openai_compat|none │
|
||||
└──────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Principles
|
||||
|
||||
1. **Offline-first.** The Android app is fully usable with no server:
|
||||
recordings live in Room + app storage; sync is a background concern.
|
||||
2. **Originals are sacred.** Uploads are stored immutably. Every processing
|
||||
step writes a new derivative asset. A failed job can never corrupt input.
|
||||
3. **Idempotent pipeline.** Jobs are keyed (recording, job_type); reruns
|
||||
overwrite derived outputs unless the user edited them; client-supplied
|
||||
`client_recording_id` makes retried uploads update-not-duplicate.
|
||||
4. **Provider independence.** Transcription and LLM live behind Protocols;
|
||||
adapters are selected by env config. `none` is a first-class provider.
|
||||
5. **UUID everywhere.** No integer IDs, no filesystem paths, no internals in
|
||||
API payloads.
|
||||
|
||||
## Processing pipeline
|
||||
|
||||
```
|
||||
upload finalize ──► [normalize (optional ffmpeg)] ──► transcribe
|
||||
│
|
||||
transcript (versioned, timestamped segments)
|
||||
│
|
||||
summarize + extract
|
||||
│
|
||||
short / detailed / key_points / decisions / action_items / questions
|
||||
```
|
||||
|
||||
Each arrow is one `processing_jobs` row: queued → running → succeeded/failed,
|
||||
with attempts and error text. Recording status aggregates the steps
|
||||
(`uploaded → queued → processing → completed | failed | ai_disabled`).
|
||||
|
||||
## Sync protocol (Android ⇄ server)
|
||||
|
||||
- Room table holds `sync_state` (LOCAL_ONLY, PENDING_UPLOAD, UPLOADING,
|
||||
SYNCED, ERROR) + `client_recording_id` (UUID, stable across retries).
|
||||
- WorkManager `UploadWorker`: constraints (Wi-Fi/charging) from user prefs;
|
||||
exponential backoff retries; manual pause cancels work, state preserved.
|
||||
- Upload = create session → PUT chunks (skip ones the session already has) →
|
||||
finalize with checksum. Resumable at chunk granularity.
|
||||
|
||||
## Why adapters-as-protocols
|
||||
|
||||
The AI layer's only job is: audio in → text out; text in → structured JSON
|
||||
out. Protocols (`sonar/ai/interfaces.py`) keep the pipeline testable with a
|
||||
`StubProvider` and make new providers (e.g. whisperX, llama.cpp) a single
|
||||
file + one env value.
|
||||
|
|
@ -23,8 +23,9 @@ following the law where you are.
|
|||
- Provides an always-visible stop control on the recording screen and in the
|
||||
notification.
|
||||
- Has no remote-trigger recording capability of any kind.
|
||||
- Collects no telemetry and sends no data anywhere except the server URL you
|
||||
configure.
|
||||
- Collects no telemetry and sends no data anywhere. There is no server,
|
||||
no account, no network permission — recordings never leave the phone
|
||||
except through file sync you set up yourself.
|
||||
|
||||
## Recommended practice
|
||||
|
||||
|
|
|
|||
|
|
@ -1,81 +0,0 @@
|
|||
# Security model (honest version)
|
||||
|
||||
This document describes what S.H.O.N.A.R. actually implements, what it does
|
||||
not, and the threat model assumptions. No claims beyond the code.
|
||||
|
||||
## Implemented
|
||||
|
||||
**Authentication**
|
||||
- Passwords hashed with Argon2id (`argon2-cffi` defaults).
|
||||
- Access tokens: short-lived (15 min) JWT HS256, `typ` claim prevents
|
||||
cross-type use.
|
||||
- Refresh tokens: opaque 384-bit values; only SHA-256 hashes stored; rotated
|
||||
on every use; token families with **reuse detection** (presenting a spent
|
||||
token revokes the entire family — commits before the 401 so the revocation
|
||||
survives the failed request).
|
||||
- Logout and device revocation revoke refresh families server-side.
|
||||
- Auth endpoints rate-limited per client IP (slowapi; `SHONAR_RATE_LIMIT_AUTH`).
|
||||
|
||||
**Authorization**
|
||||
- Every recording/transcript/summary/tag/upload-session endpoint filters by
|
||||
`user_id` server-side; ownership failures return 404 (no existence leaks).
|
||||
- User enumeration is mitigated on login (identical error for unknown email
|
||||
vs wrong password).
|
||||
|
||||
**Uploads & files**
|
||||
- Upload size caps (`SHONAR_MAX_UPLOAD_BYTES`, per-chunk caps).
|
||||
- MIME allow-list (`SHONAR_ALLOWED_AUDIO_MIME_TYPES`) + magic-byte sniffing
|
||||
on finalize (ftyp/RIFF/OggS/ID3).
|
||||
- Storage keys are server-generated UUID paths; `validate_storage_key()`
|
||||
rejects absolute paths, `..`, and any character outside `[a-z0-9-./]`;
|
||||
LocalStorage additionally verifies the resolved path stays under the root.
|
||||
- Originals are immutable: processing writes separate derivative assets.
|
||||
- File downloads go through authenticated, ownership-checked endpoints; no
|
||||
static file route exposes storage.
|
||||
|
||||
**API surface**
|
||||
- Pydantic input validation on every endpoint.
|
||||
- Global exception handler returns generic `Internal server error`; details
|
||||
are logged server-side only.
|
||||
- `/api/v1/system/status` discloses AI provider state but never secrets.
|
||||
|
||||
**Privacy**
|
||||
- No telemetry, no analytics, no crash reporting, no update pings — in the
|
||||
app or the server.
|
||||
- No network calls to AI providers unless configured; external usage is
|
||||
disclosed via `external_ai_in_use` in system status and shown in the app's
|
||||
Settings screen.
|
||||
- Location metadata is stored only when the user explicitly enables it
|
||||
(`location_storage_enabled`, default off, enforced server-side).
|
||||
|
||||
## NOT implemented (do not assume otherwise)
|
||||
|
||||
- **At-rest encryption: NOT implemented.** Files on disk and database rows
|
||||
are plaintext. Documented design if you want it:
|
||||
1. Envelope encryption: per-user data key, AES-256-GCM;
|
||||
2. DEKs wrapped by a KEK derived from a passphrase entered at app unlock
|
||||
(Argon2id), never stored server-side;
|
||||
3. Chunk-level encryption during upload so the server stores ciphertext;
|
||||
4. Trade-offs: breaks server-side transcription unless the worker holds
|
||||
keys — choose client-side-only (private but no AI) vs worker-side
|
||||
(AI works, worker is trusted).
|
||||
Until this ships, **encrypt the volume** (LUKS/zfs-encrypt) or the bucket
|
||||
(SSE) at the infrastructure layer.
|
||||
- **TLS termination:** the app enforces HTTPS for non-localhost servers, but
|
||||
the backend container serves plain HTTP — you must terminate TLS at a
|
||||
reverse proxy (example in `deploy/nginx.conf`).
|
||||
- **OIDC / 2FA:** not implemented.
|
||||
- **Account hard purge:** deletion applies a 30-day grace marker; the
|
||||
scheduled purge job is TODO (see ROADMAP). Rows and files are retained
|
||||
until it lands — do not rely on deletion as immediate erasure.
|
||||
|
||||
## Deployment assumptions
|
||||
|
||||
- Single-instance API + worker behind a trusted reverse proxy.
|
||||
- Postgres and Redis are not exposed publicly (compose binds 127.0.0.1).
|
||||
- `SHONAR_SECRET_KEY` is secret and ≥32 chars (startup warns otherwise).
|
||||
|
||||
> Accuracy note: the items under “Implemented → Uploads & files” and the
|
||||
> HTTPS enforcement in the app ship with milestones M2/M3 and are described
|
||||
> here as the design they implement; check `docs/ROADMAP.md` for what is
|
||||
> live at HEAD today.
|
||||
|
|
@ -1,198 +0,0 @@
|
|||
# Server provider architecture
|
||||
|
||||
SHONAR is an open-source mobile AI app for recording, transcribing,
|
||||
summarizing, and searching voice notes and conversations. Where your data
|
||||
lives is a **provider** decision, hard-coded nowhere in the app.
|
||||
|
||||
**Default provider: Nextcloud.** Alternatives: Start9, Umbrel, custom SHONAR
|
||||
server, or local-only storage. Home Assistant is explicitly out of initial
|
||||
scope (see `deferred/home-assistant/README.md`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Provider interface
|
||||
|
||||
One Kotlin interface (`com.shonar.provider`) — nothing else in the app talks
|
||||
to a server directly:
|
||||
|
||||
```kotlin
|
||||
interface ShonarProvider {
|
||||
val descriptor: ProviderDescriptor // id, display name, capabilities
|
||||
|
||||
// lifecycle
|
||||
suspend fun probe(baseUrl: ServerUrl): ProbeResult // server reachable & is a SHONAR service?
|
||||
suspend fun connect(credential: ProviderCredential): Unit // validate + persist (secure store)
|
||||
suspend fun reconnect(): AuthState
|
||||
suspend fun disconnect(revokeOnServer: Boolean): Unit
|
||||
suspend fun deleteAccountAndData(): Unit
|
||||
|
||||
// storage (opaque keys — provider maps to its own layout)
|
||||
suspend fun upload(recording: RecordingDraft, onProgress: (Float) -> Unit): RemoteRef
|
||||
suspend fun download(ref: RemoteRef, dest: File, onProgress: (Float) -> Unit): Unit
|
||||
suspend fun delete(ref: RemoteRef): Unit
|
||||
suspend fun list(cursor: PageCursor?): Page<RemoteRecording>
|
||||
|
||||
// syncable sidecars (transcript/summary JSON, metadata)
|
||||
suspend fun putSidecar(ref: RemoteRef, kind: SidecarKind, bytes: ByteArray): Unit
|
||||
suspend fun getSidecar(ref: RemoteRef, kind: SidecarKind): ByteArray?
|
||||
|
||||
// status shown to the user
|
||||
suspend fun storageLocationSummary(): StorageLocation // "Where is my data?" screen
|
||||
val authState: StateFlow<AuthState> // CONNECTED / EXPIRED / REVOKED / OFFLINE
|
||||
}
|
||||
```
|
||||
|
||||
`ProviderRegistry` maps `ProviderDescriptor.id -> factory`. The UI, sync
|
||||
engine, and database reference **providers by id only**. `LocalOnlyProvider`
|
||||
is a first-class implementation (no network at all), so the app never has a
|
||||
"no provider" special case.
|
||||
|
||||
## 2. Data model (Room)
|
||||
|
||||
- `recording(id UUID, title, createdAt, durationMs, localFile, mime, tags,
|
||||
notes, originProviderId, remoteRef, syncState, downloadState)`
|
||||
- `syncState`: `LOCAL_ONLY → QUEUED → UPLOADING → UPLOADED → SYNCED`, plus
|
||||
`ERROR(retryAt, reasonCode)`. Pausing/canceling is a state, not a job kill.
|
||||
- `provider(id, descriptorId, baseUrl, authKind, credentialAlias,
|
||||
certPin?, lastSeenAt, enabled)` — one row per configured provider; at most
|
||||
one `enabled` (the active one). Credential **alias** only; secrets live in
|
||||
Android Keystore-backed storage, never in Room.
|
||||
- `sidecar(recordingId, kind ∈ {TRANSCRIPT, SUMMARY, ACTION_ITEMS, KEYWORDS,
|
||||
NOTES}, contentHash, updatedAt)` — synced independently of audio.
|
||||
- Switching providers never touches `recording.localFile`; `remoteRef` +
|
||||
`originProviderId` change only, so **provider switch is lossless**.
|
||||
|
||||
## 3. Authentication flow
|
||||
|
||||
Safest-compatible-first ladder, decided per probe:
|
||||
|
||||
1. **OAuth 2 / OIDC** (Nextcloud supports it on many hosts) — authorization
|
||||
code + PKCE via Custom Tabs. App never sees the password.
|
||||
2. **Nextcloud "Login flow v2"** (`/index.php/login/v2`): server-owned
|
||||
browser consent screen, returns app password + endpoint. This is the
|
||||
default for vanilla Nextcloud and is the recommended path.
|
||||
3. **API token paste** (user already created an app password) — fallback.
|
||||
4. **Custom SHONAR server**: SHONAR's own OAuth2-compatible API (backend
|
||||
already has rotating refresh tokens, M1).
|
||||
|
||||
Rules: the user's *normal* account password is never requested, typed into
|
||||
an app screen, or stored. Tokens go to Keystore-backed storage. Token
|
||||
expiry → `EXPIRED` state with a one-tap re-auth that reuses the same flow.
|
||||
`disconnect(revokeOnServer=true)` calls the provider's revoke endpoint (e.g.
|
||||
Nextcloud `DELETE /index.php/core/auth/` via the app-password self-delete
|
||||
endpoint) before clearing local state. Account deletion goes through the
|
||||
provider API first, local wipe second.
|
||||
|
||||
## 4. TLS policy
|
||||
|
||||
- HTTPS required for any non-private host (validated in `ServerUrl`: scheme,
|
||||
host, no credentials in URL, no path traversal; private ranges per RFC
|
||||
1918/loopback allowed for LAN servers).
|
||||
- **Never** a global cert bypass. Self-signed LAN certs handled as explicit
|
||||
**trust-on-first-use**: connection attempt fails → user sees the cert
|
||||
fingerprint + hostname and must approve → pin stored (SPKI SHA-256) in
|
||||
secure storage → OkHttp pinned to that cert for that host only.
|
||||
- Cleartext HTTP permitted only for private/LAN hosts the user typed, and
|
||||
the UI shows a visible warning banner on `http://` servers.
|
||||
- Logging: OkHttp logging interceptor is **off by default** and, when a
|
||||
debug setting is enabled, redacts `Authorization`, `Cookie`, tokens, and
|
||||
bodies. Audio bytes and transcripts are never logged in any mode.
|
||||
|
||||
## 5. Provider selection UX (onboarding + settings)
|
||||
|
||||
1. First launch → consent notice (already shipped) → **Choose where your
|
||||
recordings live**:
|
||||
- **Nextcloud** *(default, marked Recommended)* — URL → probe → pick auth
|
||||
method → browser consent → connected.
|
||||
- **Start9** — URL → probe → the app discovers which SHONAR-compatible
|
||||
service is running (Server APIs are startOS app-specific; there is no
|
||||
universal Start9 storage API) → the user **identifies/confirms the
|
||||
service** from the probe result → that service's provider is bound.
|
||||
- **Umbrel** — same pattern as Start9 (Umbrel apps each expose their own
|
||||
API; probe + explicit service selection, never a universal assumption).
|
||||
- **Custom server** — URL of a SHONAR backend (`/api/v1/healthz` +
|
||||
`/api/v1/provider-info` probe identifies it as SHONAR-compatible).
|
||||
- **Local only** — nothing leaves the phone; sync features hidden,
|
||||
everything else fully functional.
|
||||
2. The chosen provider is shown persistently on Home ("Stored: Nextcloud at
|
||||
cloud.example.com"), and Settings → Server shows the full
|
||||
**StorageLocation** screen: which provider, URL, what is synced vs local,
|
||||
account, disconnect, delete account & data.
|
||||
3. Switching provider: re-run selection; existing local recordings queue
|
||||
against the new provider; old remote refs are kept in history until the
|
||||
user chooses to migrate or forget them.
|
||||
|
||||
## 6. Nextcloud provider (implement first)
|
||||
|
||||
- **APIs** (official, supported): WebDAV `PROPFIND`/`GET`/`PUT`/`MOVE`/`DELETE`
|
||||
under `/remote.php/dav/files/{user}/SHONAR/…`; login flow v2;
|
||||
`OCS /ocs/v2.php/cloud/user` for identity/quota. No proprietary calls.
|
||||
- **Layout**: `SHONAR/audio/{yyyy}/{recording-uuid}.m4a`,
|
||||
`SHONAR/sidecars/{uuid}/transcript.json` etc. Remote ref = the DAV path
|
||||
plus etag; originals are immutable (versions handled by Nextcloud's own
|
||||
versioning, never overwritten by processing artifacts).
|
||||
- **Chunked/resumable**: Nextcloud chunked-upload protocol
|
||||
(`/remote.php/dav/uploads/…`) for large recordings; WorkManager drives it
|
||||
with the same pause/resume/retry semantics already built for the SHONAR
|
||||
backend (M2 upload sessions map 1:1 onto chunked DAV uploads).
|
||||
- Probe: `GET /.well-known/webfinger` + `PROPFIND` depth 0 on the SHONAR
|
||||
folder to verify endpoint + auth before first write.
|
||||
- Auth per §3 (login flow v2 default). Quota surfaced in the StorageLocation
|
||||
screen from OCS.
|
||||
|
||||
## 7. Start9 / Umbrel / custom adapters
|
||||
|
||||
> Status: P6a shipped the generic path — Start9/Umbrel cards take a service
|
||||
> URL and auto-detect Nextcloud vs SHONAR, handing off to those providers.
|
||||
> The platform-RPC discovery below, multi-service picking, and Tor onion
|
||||
> access (Orbot SOCKS) are deferred to P6b.
|
||||
|
||||
- **Start9 & Umbrel are platforms, not APIs.** The adapter pattern is
|
||||
"platform probe + service binding": a `PlatformProbe` (Start9: Server API
|
||||
over its RPC; Umbrel: its app manifest endpoints, where available)
|
||||
enumerates *running apps*, and each app that embeds a **SHONAR-compatible
|
||||
service** (the same `/api/v1/provider-info` handshake) is offered to the
|
||||
user for explicit selection. Once selected, that binding reuses the
|
||||
custom-SHONAR-server provider. If no SHONAR-compatible service is found,
|
||||
the UI says so plainly and offers local-only or custom URL — we do **not**
|
||||
invent universal storage semantics for these platforms.
|
||||
- **Custom server** = the SHONAR FastAPI backend in this repo (already
|
||||
M0–M2): auth, chunked uploads, recordings CRUD, FTS. `provider-info`
|
||||
returns `{kind: "shonar", version, capabilities[]}` so the app can feature
|
||||
-gate (e.g. server-side transcription available?).
|
||||
- All three share the `ShonarProvider` contract; only discovery differs.
|
||||
|
||||
## 8. Test strategy
|
||||
|
||||
- **Provider contract suite** (shared, data-driven): every implementation
|
||||
(`LocalOnly`, `CustomShonar`, `Nextcloud`) runs the same test list against
|
||||
MockWebServer / fake FS — upload/resume/pause/retry, sidecars, delete,
|
||||
auth-state transitions (CONNECTED→EXPIRED→re-auth→CONNECTED), revocation,
|
||||
cert-pin TOFU, URL validation rejects (cleartext public host, credentials
|
||||
in URL, traversal).
|
||||
- **Nextcloud-specific**: login-flow v2 handshake, DAV path layout, chunked
|
||||
upload protocol, quota parsing — against recorded MockWebServer fixtures.
|
||||
- **Leak tests**: assert no credential, token, audio byte, or transcript
|
||||
string ever appears in app logs (logcat capture under debug setting
|
||||
enabled + redaction assertions) and in Room tables.
|
||||
- **Local-only invariant**: device in airplane mode → full record/playback/
|
||||
search functionality passes.
|
||||
- Backend keeps its pytest suite (26 green as of M2). Android unit tests
|
||||
keep 19 settings tests green; provider tests added per phase.
|
||||
|
||||
## 9. Phased plan
|
||||
|
||||
| Phase | Deliverable | Verify |
|
||||
|---|---|---|
|
||||
| P0 (done) | HA isolated to `deferred/` + branch; removed from build, onboarding, defaults, tests | build + unit tests green |
|
||||
| P1 | `provider/` module: `ShonarProvider` interface, `ServerUrl` validation, `LocalOnlyProvider`, `ProviderRegistry` | contract suite (local-only) |
|
||||
| P2 | Provider-selection onboarding screen + StorageLocation screen wired to registry | on-device |
|
||||
| P3 (done) | `CustomShonarProvider` against this repo's backend (auth M1 + uploads M2), WorkManager sync states | contract suite (10 shared + 12 custom) + 9 auth tests vs MockWebServer fixtures mirroring `schemas_*.py`; backend pytest 28 green on live Postgres; `assembleDebug` clean |
|
||||
| P4 (done) | `NextcloudProvider`: login flow v2, DAV upload/download/delete, chunking — plus `FolderSyncProvider` (sync-folder for Syncthing etc., requested alongside) | contract suite (10 shared + 6 custom) + 6 auth + 4 protocol tests vs MockWebServer fake; DAV/chunking-v2 protocol verified live against Nextcloud 34 (MKCOL/PUT/MOVE-assemble/PROPFIND/OCS/login-v2 via curl); full suite 120 green, `assembleDebug` clean. On-device browser-approval tap-through still pending |
|
||||
| P5 (done) | TLS TOFU pinning + redaction/logging + leak tests | TOFU trust manager (system-first, per-host DER pins in secure store, explicit approval UI with fingerprint + issuer + validity) + redacting logger gated by `log_http_bodies` (transcript/sidecar bodies never log, in any mode) + HeldCertificate TLS fixtures; full suite 138 green, `assembleDebug` clean. Self-signed hosts now connect after one approval; hostname verification stays strict |
|
||||
| P6a (done) | Generic hosted setup: Start9/Umbrel cards take a service URL, auto-detect Nextcloud vs SHONAR, hand off to the existing provider flows; the platform is an entry path, the persisted provider is the protocol | routing matrix unit tests (`routeHosted`); full suite + `assembleDebug` |
|
||||
| P6b (deferred) | Platform RPC auto-discovery (Start9 Server API, Umbrel manifests), multi-service picker, Tor onion access via Orbot SOCKS | probe fakes + on-device, when needed |
|
||||
| P7 (done) | Provider switching w/ migration prompts, account deletion, revocation | Room v2 provider slot per recording (origin + remote ref + sync state, honest migration) + "Where is my data?" screen (summary, reconnect, disconnect+revoke, delete account) + foreground migrate-now uploader (bounded: cancellable, one attempt/file, upload-only) + switch-time prompt (upload / keep / forget links); migration matrix + slots + converter tests, full suite 156 green, `assembleDebug` clean. Background/scheduled sync stays M5; pull-down and old-remote wipe are explicit non-goals (a future wipe is safe: everything lives under `SHONAR/` prefixes) |
|
||||
|
||||
Recording engine (M4) and playback (M6) proceed independently on top of the
|
||||
same Room model; provider work is orthogonal.
|
||||
Loading…
Add table
Add a link
Reference in a new issue