Product pivot: defer Home Assistant; provider-based server architecture (Nextcloud default)
ISOLATE (nothing deleted): - moved HA module (ha/, ui/devices/, HA client tests), e2e scripts, and HA docs under deferred/home-assistant/ with a README explaining status + how to revive; complete snapshot preserved on branch deferred/home-assistant REMOVE FROM ACTIVE PRODUCT: - HomeScreen: Devices card + route gone; MainActivity nav updated - ShonarApplication: haRepository removed - BuiltInSettings: Home Assistant category/settings removed from defaults - SettingsManagerTest: secret tests rewritten around a user-created SECRET-type setting (no built-in secret ships) - Manifest + URL-validation test fixture wording neutralized - README/ROADMAP: HA marked deferred with pointer to preserved branch ADD (design, per product direction): - docs/server-providers.md: ShonarProvider interface, Room data model, auth ladder (OIDC/PKCE -> Nextcloud login-flow-v2 -> token paste), sync strategy, provider-selection UX (Nextcloud default; Start9/Umbrel as platform-probe + explicit service binding, never universal APIs; custom SHONAR server; local-only), TLS TOFU pinning policy, no-secret-logging rules, provider contract test strategy, phased plan P0-P7 VERIFY: 19 Android unit tests green, APK builds, on-device launch OK (consent dialog renders; no Devices entry). Backend unchanged (26 tests).
This commit is contained in:
avi
parent
978ca908e1
commit
4eab1f11cf
22 changed files with 289 additions and 87 deletions
|
|
@ -10,8 +10,8 @@ updated in the same commit as the work it describes.
|
|||
| M1 | Auth: register / login / rotating refresh + reuse detection / logout / delete-account, Argon2id, rate limits | done |
|
||||
| M2 | Upload sessions (chunked, resumable), storage abstraction (local + S3), recordings CRUD, ownership checks | done |
|
||||
| M3 | Android: server URL config, login, token persistence + auto-refresh | TODO |
|
||||
| HA-1 | Android: generic Custom Settings engine (8 types, validation, custom CRUD, import/export, secure storage) + Home Assistant integration (REST + WebSocket, discovery, service calls, reconnect) — 29 unit tests green, APK builds | done |
|
||||
| HA-2 | Home Assistant live end-to-end against a real server (token minting flow blocked; unit + MockWebServer coverage only) | in progress |
|
||||
| S-1 | Android: generic Custom Settings engine (8 types, validation, custom CRUD, import/export, secure storage) — 19 unit tests green | done |
|
||||
| ~~HA-1/HA-2~~ | Home Assistant integration (client, repository, devices screen, e2e scripts) | **DEFERRED — out of initial product scope; preserved under `deferred/home-assistant/` and branch `deferred/home-assistant`** |
|
||||
| M4 | Android: foreground-service recording (pause/resume/stop), metadata, Room | TODO |
|
||||
| M5 | Android: WorkManager upload sync (retry, Wi-Fi-only, charging-only, pause) | TODO |
|
||||
| M6 | Android: library (search/filter/sort), playback (seek/speed), waveform, download/delete | TODO |
|
||||
|
|
|
|||
|
|
@ -1,106 +0,0 @@
|
|||
# Home Assistant integration — guide
|
||||
|
||||
SHONAR can talk to a **local** Home Assistant server over Home Assistant's
|
||||
official REST and WebSocket APIs. No cloud, no third-party wrappers, no
|
||||
telemetry — the app talks only to the URL you configure.
|
||||
|
||||
## 1. Install Home Assistant locally
|
||||
|
||||
Any of these works; see <https://www.home-assistant.io/installation/>:
|
||||
|
||||
- **Home Assistant OS** on a Raspberry Pi / VM (recommended)
|
||||
- **Container:** `docker run -d --name homeassistant --restart=unless-stopped \
|
||||
-v ./config:/config -e TZ=UTC --network=host ghcr.io/home-assistant/home-assistant:stable`
|
||||
- **Core:** Python venv install (advanced)
|
||||
|
||||
After first start the UI is at `http://<host>:8123`.
|
||||
|
||||
## 2. Create a long-lived access token
|
||||
|
||||
Home Assistant requires a long-lived access token for third-party apps
|
||||
(SHONAR cannot mint one for you — that's an HA security design):
|
||||
|
||||
1. Open the HA web UI and log in.
|
||||
2. Click your **profile picture** (bottom-left) → **Security**.
|
||||
3. Scroll to **Long-lived access tokens** → **Create Token**.
|
||||
4. Name it (e.g. `SHONAR`) and **copy the token immediately** — it is shown
|
||||
once.
|
||||
|
||||
## 3. Where to enter the URL and token
|
||||
|
||||
SHONAR app → **Settings** → **Home Assistant** category:
|
||||
|
||||
1. Toggle **Enable Home Assistant** on.
|
||||
2. **Home Assistant URL** — e.g. `http://homeassistant.local:8123` or
|
||||
`http://192.168.1.50:8123`.
|
||||
3. **Long-lived access token** — paste the token. Stored encrypted on the
|
||||
device (AndroidKeyStore-backed EncryptedSharedPreferences); never in
|
||||
plaintext, never logged, never included in settings export unless you
|
||||
explicitly request it.
|
||||
|
||||
The **Devices** tab then shows your entities. The connection status appears
|
||||
there: not configured / connecting / connected / error (with a plain-language
|
||||
message you can act on).
|
||||
|
||||
## 4. Test the connection
|
||||
|
||||
- Settings → Home Assistant has the URL/token fields; the Devices screen
|
||||
doubles as the live connection test — hit the refresh icon.
|
||||
- Or from a shell on the same network:
|
||||
`curl -H "Authorization: Bearer <token>" http://<ha-host>:8123/api/`
|
||||
|
||||
## 5. How entity discovery works
|
||||
|
||||
`GET /api/states` returns every entity; the app lists them with friendly
|
||||
names, current state, and a search box. Entities whose domain is switchable
|
||||
(`light`, `switch`, `fan`, `input_boolean`, `humidifier`) get a toggle that
|
||||
calls `POST /api/services/<domain>/turn_on|turn_off` with `entity_id`.
|
||||
Toggles apply optimistically and are reconciled by the live stream.
|
||||
|
||||
Live updates: a WebSocket to `/api/websocket` authenticates and subscribes to
|
||||
`state_changed` events. On disconnect it reconnects with exponential backoff
|
||||
(2s → 60s). A configurable poll fallback (`State refresh interval`, used when
|
||||
the WebSocket is down) keeps the list fresh.
|
||||
|
||||
## 6. How custom settings work
|
||||
|
||||
All settings are data-driven (`SettingDefinition`), not hard-coded UI. Each
|
||||
has an id, name, description, category, type (boolean/string/number/select/
|
||||
multi-select/color/url/secret), default, optional min/max/choices,
|
||||
editability, sensitivity, restart requirement, and conditional visibility.
|
||||
|
||||
Settings → **Add (+)** creates a custom setting with any type; **delete**
|
||||
works on custom settings only (built-ins can be reset but not removed).
|
||||
Search filters across name/id/description. **Export** produces JSON with
|
||||
secrets omitted by default; **Import** validates every entry against known
|
||||
definitions and applies all-or-nothing, reporting per-key rejection reasons.
|
||||
|
||||
## 7. How settings are stored
|
||||
|
||||
| Kind | Location |
|
||||
|---|---|
|
||||
| Ordinary values + custom definitions | Jetpack DataStore (`shonar_settings`) |
|
||||
| Sensitive values (HA token, secret-type settings) | EncryptedSharedPreferences, AES256-GCM values / AES256-SIV keys, master key in AndroidKeyStore (never exportable) |
|
||||
|
||||
## 8. Troubleshooting
|
||||
|
||||
| Symptom | Likely cause / fix |
|
||||
|---|---|
|
||||
| "Authentication failed…" | Token expired/revoked/typo. Create a fresh long-lived access token (step 2). |
|
||||
| "Cannot reach the Home Assistant server" | Wrong URL/IP, phone on different network/VLAN, HA not running, or a firewall blocking 8123. Try the curl test above from the phone's network. |
|
||||
| Cleartext HTTP warning | The app permits cleartext HTTP (like the official Home Assistant Android app) because LAN servers are plain `http://` by default. Android cannot restrict cleartext to IP *ranges* — use `https://` for any non-LAN host; certificate verification is never disabled. |
|
||||
| Entities stale | WebSocket dropped; poll fallback updates after the configured interval; refresh icon forces a reload. |
|
||||
| mDNS name not resolving | Use the IP address instead of `homeassistant.local` (phone's mDNS may be limited by the router). |
|
||||
|
||||
## 9. Endpoints used (official API only)
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| GET | `/api/` | connection/config test |
|
||||
| GET | `/api/states` | entity discovery |
|
||||
| GET | `/api/states/{entity_id}` | single state |
|
||||
| POST | `/api/services/{domain}/{service}` | service/action calls |
|
||||
| WS | `/api/websocket` | `auth`, `subscribe_events(state_changed)` |
|
||||
|
||||
References: REST <https://developers.home-assistant.io/docs/api/rest/>,
|
||||
WebSocket <https://developers.home-assistant.io/docs/api/websocket/>.
|
||||
192
docs/server-providers.md
Normal file
192
docs/server-providers.md
Normal file
|
|
@ -0,0 +1,192 @@
|
|||
# Server provider architecture
|
||||
|
||||
SHONAR is an open-source mobile AI app for recording, transcribing,
|
||||
summarizing, and searching voice notes and conversations. Where your data
|
||||
lives is a **provider** decision, hard-coded nowhere in the app.
|
||||
|
||||
**Default provider: Nextcloud.** Alternatives: Start9, Umbrel, custom SHONAR
|
||||
server, or local-only storage. Home Assistant is explicitly out of initial
|
||||
scope (see `deferred/home-assistant/README.md`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Provider interface
|
||||
|
||||
One Kotlin interface (`com.shonar.provider`) — nothing else in the app talks
|
||||
to a server directly:
|
||||
|
||||
```kotlin
|
||||
interface ShonarProvider {
|
||||
val descriptor: ProviderDescriptor // id, display name, capabilities
|
||||
|
||||
// lifecycle
|
||||
suspend fun probe(baseUrl: ServerUrl): ProbeResult // server reachable & is a SHONAR service?
|
||||
suspend fun connect(credential: ProviderCredential): Unit // validate + persist (secure store)
|
||||
suspend fun reconnect(): AuthState
|
||||
suspend fun disconnect(revokeOnServer: Boolean): Unit
|
||||
suspend fun deleteAccountAndData(): Unit
|
||||
|
||||
// storage (opaque keys — provider maps to its own layout)
|
||||
suspend fun upload(recording: RecordingDraft, onProgress: (Float) -> Unit): RemoteRef
|
||||
suspend fun download(ref: RemoteRef, dest: File, onProgress: (Float) -> Unit): Unit
|
||||
suspend fun delete(ref: RemoteRef): Unit
|
||||
suspend fun list(cursor: PageCursor?): Page<RemoteRecording>
|
||||
|
||||
// syncable sidecars (transcript/summary JSON, metadata)
|
||||
suspend fun putSidecar(ref: RemoteRef, kind: SidecarKind, bytes: ByteArray): Unit
|
||||
suspend fun getSidecar(ref: RemoteRef, kind: SidecarKind): ByteArray?
|
||||
|
||||
// status shown to the user
|
||||
suspend fun storageLocationSummary(): StorageLocation // "Where is my data?" screen
|
||||
val authState: StateFlow<AuthState> // CONNECTED / EXPIRED / REVOKED / OFFLINE
|
||||
}
|
||||
```
|
||||
|
||||
`ProviderRegistry` maps `ProviderDescriptor.id -> factory`. The UI, sync
|
||||
engine, and database reference **providers by id only**. `LocalOnlyProvider`
|
||||
is a first-class implementation (no network at all), so the app never has a
|
||||
"no provider" special case.
|
||||
|
||||
## 2. Data model (Room)
|
||||
|
||||
- `recording(id UUID, title, createdAt, durationMs, localFile, mime, tags,
|
||||
notes, originProviderId, remoteRef, syncState, downloadState)`
|
||||
- `syncState`: `LOCAL_ONLY → QUEUED → UPLOADING → UPLOADED → SYNCED`, plus
|
||||
`ERROR(retryAt, reasonCode)`. Pausing/canceling is a state, not a job kill.
|
||||
- `provider(id, descriptorId, baseUrl, authKind, credentialAlias,
|
||||
certPin?, lastSeenAt, enabled)` — one row per configured provider; at most
|
||||
one `enabled` (the active one). Credential **alias** only; secrets live in
|
||||
Android Keystore-backed storage, never in Room.
|
||||
- `sidecar(recordingId, kind ∈ {TRANSCRIPT, SUMMARY, ACTION_ITEMS, KEYWORDS,
|
||||
NOTES}, contentHash, updatedAt)` — synced independently of audio.
|
||||
- Switching providers never touches `recording.localFile`; `remoteRef` +
|
||||
`originProviderId` change only, so **provider switch is lossless**.
|
||||
|
||||
## 3. Authentication flow
|
||||
|
||||
Safest-compatible-first ladder, decided per probe:
|
||||
|
||||
1. **OAuth 2 / OIDC** (Nextcloud supports it on many hosts) — authorization
|
||||
code + PKCE via Custom Tabs. App never sees the password.
|
||||
2. **Nextcloud "Login flow v2"** (`/index.php/login/v2`): server-owned
|
||||
browser consent screen, returns app password + endpoint. This is the
|
||||
default for vanilla Nextcloud and is the recommended path.
|
||||
3. **API token paste** (user already created an app password) — fallback.
|
||||
4. **Custom SHONAR server**: SHONAR's own OAuth2-compatible API (backend
|
||||
already has rotating refresh tokens, M1).
|
||||
|
||||
Rules: the user's *normal* account password is never requested, typed into
|
||||
an app screen, or stored. Tokens go to Keystore-backed storage. Token
|
||||
expiry → `EXPIRED` state with a one-tap re-auth that reuses the same flow.
|
||||
`disconnect(revokeOnServer=true)` calls the provider's revoke endpoint (e.g.
|
||||
Nextcloud `DELETE /index.php/core/auth/` via the app-password self-delete
|
||||
endpoint) before clearing local state. Account deletion goes through the
|
||||
provider API first, local wipe second.
|
||||
|
||||
## 4. TLS policy
|
||||
|
||||
- HTTPS required for any non-private host (validated in `ServerUrl`: scheme,
|
||||
host, no credentials in URL, no path traversal; private ranges per RFC
|
||||
1918/loopback allowed for LAN servers).
|
||||
- **Never** a global cert bypass. Self-signed LAN certs handled as explicit
|
||||
**trust-on-first-use**: connection attempt fails → user sees the cert
|
||||
fingerprint + hostname and must approve → pin stored (SPKI SHA-256) in
|
||||
secure storage → OkHttp pinned to that cert for that host only.
|
||||
- Cleartext HTTP permitted only for private/LAN hosts the user typed, and
|
||||
the UI shows a visible warning banner on `http://` servers.
|
||||
- Logging: OkHttp logging interceptor is **off by default** and, when a
|
||||
debug setting is enabled, redacts `Authorization`, `Cookie`, tokens, and
|
||||
bodies. Audio bytes and transcripts are never logged in any mode.
|
||||
|
||||
## 5. Provider selection UX (onboarding + settings)
|
||||
|
||||
1. First launch → consent notice (already shipped) → **Choose where your
|
||||
recordings live**:
|
||||
- **Nextcloud** *(default, marked Recommended)* — URL → probe → pick auth
|
||||
method → browser consent → connected.
|
||||
- **Start9** — URL → probe → the app discovers which SHONAR-compatible
|
||||
service is running (Server APIs are startOS app-specific; there is no
|
||||
universal Start9 storage API) → the user **identifies/confirms the
|
||||
service** from the probe result → that service's provider is bound.
|
||||
- **Umbrel** — same pattern as Start9 (Umbrel apps each expose their own
|
||||
API; probe + explicit service selection, never a universal assumption).
|
||||
- **Custom server** — URL of a SHONAR backend (`/api/v1/healthz` +
|
||||
`/api/v1/provider-info` probe identifies it as SHONAR-compatible).
|
||||
- **Local only** — nothing leaves the phone; sync features hidden,
|
||||
everything else fully functional.
|
||||
2. The chosen provider is shown persistently on Home ("Stored: Nextcloud at
|
||||
cloud.example.com"), and Settings → Server shows the full
|
||||
**StorageLocation** screen: which provider, URL, what is synced vs local,
|
||||
account, disconnect, delete account & data.
|
||||
3. Switching provider: re-run selection; existing local recordings queue
|
||||
against the new provider; old remote refs are kept in history until the
|
||||
user chooses to migrate or forget them.
|
||||
|
||||
## 6. Nextcloud provider (implement first)
|
||||
|
||||
- **APIs** (official, supported): WebDAV `PROPFIND`/`GET`/`PUT`/`MOVE`/`DELETE`
|
||||
under `/remote.php/dav/files/{user}/SHONAR/…`; login flow v2;
|
||||
`OCS /ocs/v2.php/cloud/user` for identity/quota. No proprietary calls.
|
||||
- **Layout**: `SHONAR/audio/{yyyy}/{recording-uuid}.m4a`,
|
||||
`SHONAR/sidecars/{uuid}/transcript.json` etc. Remote ref = the DAV path
|
||||
plus etag; originals are immutable (versions handled by Nextcloud's own
|
||||
versioning, never overwritten by processing artifacts).
|
||||
- **Chunked/resumable**: Nextcloud chunked-upload protocol
|
||||
(`/remote.php/dav/uploads/…`) for large recordings; WorkManager drives it
|
||||
with the same pause/resume/retry semantics already built for the SHONAR
|
||||
backend (M2 upload sessions map 1:1 onto chunked DAV uploads).
|
||||
- Probe: `GET /.well-known/webfinger` + `PROPFIND` depth 0 on the SHONAR
|
||||
folder to verify endpoint + auth before first write.
|
||||
- Auth per §3 (login flow v2 default). Quota surfaced in the StorageLocation
|
||||
screen from OCS.
|
||||
|
||||
## 7. Start9 / Umbrel / custom adapters
|
||||
|
||||
- **Start9 & Umbrel are platforms, not APIs.** The adapter pattern is
|
||||
"platform probe + service binding": a `PlatformProbe` (Start9: Server API
|
||||
over its RPC; Umbrel: its app manifest endpoints, where available)
|
||||
enumerates *running apps*, and each app that embeds a **SHONAR-compatible
|
||||
service** (the same `/api/v1/provider-info` handshake) is offered to the
|
||||
user for explicit selection. Once selected, that binding reuses the
|
||||
custom-SHONAR-server provider. If no SHONAR-compatible service is found,
|
||||
the UI says so plainly and offers local-only or custom URL — we do **not**
|
||||
invent universal storage semantics for these platforms.
|
||||
- **Custom server** = the SHONAR FastAPI backend in this repo (already
|
||||
M0–M2): auth, chunked uploads, recordings CRUD, FTS. `provider-info`
|
||||
returns `{kind: "shonar", version, capabilities[]}` so the app can feature
|
||||
-gate (e.g. server-side transcription available?).
|
||||
- All three share the `ShonarProvider` contract; only discovery differs.
|
||||
|
||||
## 8. Test strategy
|
||||
|
||||
- **Provider contract suite** (shared, data-driven): every implementation
|
||||
(`LocalOnly`, `CustomShonar`, `Nextcloud`) runs the same test list against
|
||||
MockWebServer / fake FS — upload/resume/pause/retry, sidecars, delete,
|
||||
auth-state transitions (CONNECTED→EXPIRED→re-auth→CONNECTED), revocation,
|
||||
cert-pin TOFU, URL validation rejects (cleartext public host, credentials
|
||||
in URL, traversal).
|
||||
- **Nextcloud-specific**: login-flow v2 handshake, DAV path layout, chunked
|
||||
upload protocol, quota parsing — against recorded MockWebServer fixtures.
|
||||
- **Leak tests**: assert no credential, token, audio byte, or transcript
|
||||
string ever appears in app logs (logcat capture under debug setting
|
||||
enabled + redaction assertions) and in Room tables.
|
||||
- **Local-only invariant**: device in airplane mode → full record/playback/
|
||||
search functionality passes.
|
||||
- Backend keeps its pytest suite (26 green as of M2). Android unit tests
|
||||
keep 19 settings tests green; provider tests added per phase.
|
||||
|
||||
## 9. Phased plan
|
||||
|
||||
| Phase | Deliverable | Verify |
|
||||
|---|---|---|
|
||||
| P0 (done) | HA isolated to `deferred/` + branch; removed from build, onboarding, defaults, tests | build + unit tests green |
|
||||
| P1 | `provider/` module: `ShonarProvider` interface, `ServerUrl` validation, `LocalOnlyProvider`, `ProviderRegistry` | contract suite (local-only) |
|
||||
| P2 | Provider-selection onboarding screen + StorageLocation screen wired to registry | on-device |
|
||||
| P3 | `CustomShonarProvider` against this repo's backend (auth M1 + uploads M2), WorkManager sync states | backend + app integration |
|
||||
| P4 | `NextcloudProvider`: login flow v2, DAV upload/download/delete, chunking | contract suite + a real Nextcloud instance |
|
||||
| P5 | TLS TOFU pinning + redaction/logging + leak tests | cert fixtures |
|
||||
| P6 | Platform probes for Start9/Umbrel + service-binding UX | probe fakes |
|
||||
| P7 | Provider switching w/ migration prompts, account deletion, revocation | e2e |
|
||||
|
||||
Recording engine (M4) and playback (M6) proceed independently on top of the
|
||||
same Room model; provider work is orthogonal.
|
||||
Loading…
Add table
Add a link
Reference in a new issue