Commit graph

101 commits

Author SHA1 Message Date
avi
6ec811393a Migrations no longer clobber app logging: drop fileConfig(alembic.ini) from migrations/env.py. fileConfig rewrote the root logger (WARN + disable_existing) during the engine's startup migration, silently muting every INFO line afterwards — engine.log froze at the migration's last line and uvicorn access logs vanished, swallowing the TRACE instrumentation. The app owns logging (main.py basicConfig); alembic.ini's [loggers] applied only to standalone CLI runs. Verified: fresh engine boot now logs access lines live (GET /jobs flowing), 83 tests pass. 2026-09-18 13:13:40 -05:00
avi
768b75915a Instrumentation only: TRACE log lines for the tone-mismatch + progress investigation — reprocess (rid, requested tone, prior tone/status), summarize-start (rid, job, tone the worker actually read), progress (every distinct pct the provider emits), store_summary (rid, tone saved). rid minted per click joins all lines; zero behavior change. 2026-09-18 10:49:28 -05:00
avi
e8bd5623d4 PRODUCT.md: impeccable product record — local-first promise, four fixed voices, honest-status principles; sonar look incumbent but not binding 2026-09-18 09:46:11 -05:00
avi
54e1f90c22 Detail: upload+transcribe now polls the WHOLE pipeline (transcribe AND summarize) before leaving the live-update loop — breaking on transcribe alone left 'Summarizing… 0%' on screen after the summary actually finished (seen live on 'Ali Tall buildings': engine succeeded at 13:38:52, screen kept the stale job snapshot until reopen). Also clears the busy label when no job is active instead of leaving the last one. 2026-09-18 09:10:45 -05:00
avi
bcb32e4f01 Detail playback row: drop the redundant Pause/Resume pill and the duplicate Stop text button — one filled Stop button, always present while playing 2026-09-18 08:33:43 -05:00
avi
5d1f615d4a Snapshot: summary-notify compiles + SummaryNotifyTest (compile+tests green) 2026-09-17 22:23:35 -05:00
avi
add622d649 WIP snapshot: summary-ready toast + unseen badge (safety commit before power loss; sibling session actively editing — may not compile yet) 2026-09-17 22:17:59 -05:00
avi
3061629dc5 Summarize: LAN primary with local-Ollama rescue + in-app failure guidance
- Engine: when the primary summarizer is out of retries (or misconfigured),
  run_summarize now finishes the job on the configured rescue provider
  (SHONAR_LLM_FALLBACK_*), tags the summary with the provider that wrote it,
  and stores a human note in job.error; success clears stale notes.
- App (auto/lan): passes Ollama as the rescue provider when it is up.
- Detail screen: shows the rescue-swap note in plain words, a red
  'Summary failed' line with Settings -> Summarizer fix instructions and a
  Retry summary button on hard failure.
- Settings copy explains the fallback. 3 new pytest cases (14/14 pass);
  live E2E on 2026-09-18: sarcastic summary v6 via LAN on attempt 2.
2026-09-17 21:44:02 -05:00
avi
cb0492fb3e Fix stuck 'Summarizing… 0%': the 'already running — attach to it' branch in reprocessInPlace returned without polling, stranding the initial busy label on screen forever (comment promised live progress the code never did). Now it falls through to the same poll loop as a fresh job. Also makes every early-return clear the busy state. 2026-09-17 16:16:27 -05:00
avi
1eb6d55f4d Honest summarize progress: backend percent pins at 99 the moment tokens start (thinking models stream immediately) then holds there for minutes — so streaming now shows 'Summarizing… working · m:ss' with an indeterminate bar; '0% waiting for model' stays for the pre-token wait; 100 only on success. Transcribe keeps its real percentage. 2026-09-17 14:43:03 -05:00
avi
8586bc2476 Summarize always shows a percentage: the pre-token wait state read 'Summarizing… waiting for model' with no number; now 'Summarizing… 0% waiting for model' everywhere it appears (jobProgress, both detail labels, reprocess initial busy). Tests updated. 2026-09-17 14:26:16 -05:00
avi
50d5a0c982 Library filter chips: drop Done and Failed (user request — All and Untranscribed cover the actionable cases; filter logic untouched). 2026-09-17 14:12:36 -05:00
avi
26f7b273ce Never double-import a file into the engine: detail Transcribe bails when the pump already owns the file (in-flight check was absent there — mid-upload the mapping sidecar doesn't exist yet, so the upload path created a second server recording; happened live with Gambino Tahoe 11-50). Pump's transcribeFile reuses an existing mapping via /reprocess instead of uploading fresh (a pipeline killed mid-run no longer re-imports as a new recording on next pump). Purged the 11:08 orphan recording from the engine DB. 2026-09-17 13:38:01 -05:00
avi
1f688e79bc Voice chips directly under the summary (before the long sections): every position further down fell off-screen and read as 'missing'; saved-report path keeps them under the report; dedupe with the bottom block 2026-09-17 11:09:19 -05:00
avi
104aaa4d66 Detail/library UX pass from live-use feedback:
- Detail: pinned header (leave from anywhere), Prev/Next + Alt+arrows + Esc
- Summary above transcript with full sections (key points/decisions/action
  items/details) + Copy summary / Copy action items buttons
- Voice chips moved under the summary; one-page scroll below the header
- Stuck-at-99% fixed: detail re-fetches transcript+summary when a watched
  job finishes while the screen is open
- Playback: Pause/Resume + seek slider + position readout over mpv IPC
  (socket verified; controls hide if IPC unavailable)
- Library: sort Name/Newest/Oldest (persisted, date shown per row) and
  status filter All/Untranscribed/Done/Failed
- Delete is undoable: trash + 10s Undo toast replaces confirm dialogs
  (single + bulk); startup sweeps stale trash; engine purge on undo-window
  close
2026-09-17 10:00:03 -05:00
avi
134742fc67 STATUS: 2026-09-16 open-item audit — Large v3 first-use download verified end-to-end in an isolated HF cache (2.9 GB snapshot, model loads + transcribes); skipped test resolved as intentional; adoption/mapping items already closed 2026-09-16 20:16:57 -05:00
avi
0a0702f15d Summarize progress counts reasoning tokens: thinking models stream most of the run in reasoning_content, which the ticker ignored — UI sat at 'waiting for model' then jumped to done with no percentage 2026-09-16 18:28:59 -05:00
avi
8f7e1f9ea5 Detail: voice chip click re-summarizes immediately; separate Re-summarize button removed (redundant) 2026-09-16 17:28:42 -05:00
avi
25d53cb251 Detail: always fetch live summary for open file — voice tag was invisible for library-opened files (the report's own '## Summary' section defeated the fetch heuristic) 2026-09-16 17:12:10 -05:00
avi
3e1ec8eff8 Queue wait is visible: 'Queued… · Ns' + created_at on job payloads 2026-09-16 16:15:32 -05:00
avi
1dde1019e9 Detail: unmapable Summarize/transcript-save say why instead of silently doing nothing 2026-09-16 11:57:36 -05:00
avi
5ed4f31973 Always state the summary's voice: tag on the detail screen, Voice line in the saved report 2026-09-16 11:36:50 -05:00
avi
d66c27463c Settings: pick the summarizer (LAN H200 / laptop Ollama / auto); waiting label shows elapsed wait 2026-09-16 11:26:15 -05:00
avi
8e465bdb93 Detail: edit the transcript in place (user version wins over re-transcribe) 2026-09-16 11:26:15 -05:00
avi
ba54cb853b Adopt engine jobs at startup and on 'already running'; live work outranks a saved report 2026-09-15 19:45:02 -05:00
avi
7fa99e8291 Summarize streams with real progress: adapters reassemble SSE/NDJSON deltas and report 0-99% as tokens arrive 2026-09-15 19:45:02 -05:00
avi
cd1b08fda0 Summarize voices: exactly the four personas, drop custom-voice chip 2026-09-15 15:40:19 -05:00
avi
7d3e0e886d Detail: inline summarize progress + more voices + custom voice
- Progress bar + 'Summarizing… · <voice> (attempt n/3)' right under the
  Re-summarize button: the top spinner hides behind a long report, which
  made a running job look like nothing happened.
- Voices: + Enthusiastic, + Storyteller, + 'Other…' free-text (any
  phrase, 60 chars; dialog shows how it reads in the prompt).
- Verified end-to-end on BOTS: sarcastic summarize succeeded, summary
  v2 stored with tone=sarcastic ('…a heroic inability to monetize
  anything'). BUILD SUCCESSFUL + 39 tests.
2026-09-15 15:10:39 -05:00
avi
994dbc3690 Detail: personality picker beside Re-summarize
Chips: Neutral / Sarcastic / Funny / Dry wit. The chosen voice rides
the reprocess call and the button echoes it ('Re-summarize · Sarcastic');
an existing summary shows '· in a sarcastic voice' next to its heading.
compileKotlin + 39 unit tests green.
2026-09-15 14:24:36 -05:00
avi
b63f49241d Summarize tone: re-summarize in a chosen voice (backend)
POST /reprocess?job=summarize&tone=<voice> persists the voice on the job
row (queue carries only ids, so a sweep re-enqueue keeps it), the LLM
prompt appends 'write every field in a <tone> tone — the tone colors
the wording, never the facts', and the resulting summary records its
tone (SummaryOut.tone). Plain re-summarize clears a previous tone.
Migration tone0000000001 (summaries.tone, processing_jobs.tone).
78 passed, 1 skipped; ruff clean.
2026-09-15 14:24:36 -05:00
avi
0d8f3be614 Settings screen: make root column scrollable
Model cards + folder row overflowed in short windows (split-screen) with
no scroll path — the only screen where content was truly unreachable.
Library/Detail already scroll via LazyColumns; verified compileKotlin +
39 unit tests green.
2026-09-15 14:00:25 -05:00
avi
2a1d5d5ab2 Add STATUS.md checkpoint: verified state, architecture decisions, open items
Folds in the Sep 14-15 cycle (Android removal, batch transcribe, library
bulk ops, Debian path) with live test counts and the five open candidates.
2026-09-15 12:47:10 -05:00
avi
6599935e28 Config: drop unused search_backend knob
Nothing read it; services/search.py already picks SQLite vs Postgres by
dialect. The 'postgres_fts' default was a leftover from the server-era
config and only invited confusion.
2026-09-15 12:46:18 -05:00
avi
0b778e179d Desktop: drop per-row Delete; selection bar's Delete is the one path
Every row had a Delete button that duplicated the bulk bar (check a
box -> Delete N). Removed it plus its now-dead single-file confirm
dialog; deletion is checkbox + bar from now on. Detail screen keeps
its own delete.
2026-09-15 12:11:33 -05:00
avi
71b55715b4 Remove all Android/phone code: the app is standalone desktop
shared/ carried the vendored phone app's tree behind a 30-entry
exclude list. Deleted the 46 phone-only files (recording service,
widgets, Room DB, Nextcloud/local-only/folder-sync providers, TOFU
trust, Android settings stores, phone UI screens) and kept only the
seven the desktop compiles: CustomShonarProvider, ProviderTypes,
ShonarProvider, ShonarAuthStore, ShonarHandshake, ProviderRegistry,
AiContent. ProviderRegistry reduces to its id constant; handshake
drops the TOFU param; provider defaults self-identify as desktop.
Gradle exclude list gone — srcDir('../shared') is pure desktop now.
2026-09-15 11:42:20 -05:00
avi
751428cf3e Desktop: batch card shows overall % plus current file's stage and % 2026-09-15 10:53:58 -05:00
avi
36a1d2374b Desktop: queued files can be deleted/renamed; only running jobs lock
inFlight conflated queued with actually-running, so a file waiting in
the pump refused deletes ('is transcribing' — false) and hid its
Rename/Delete buttons during a batch. Track the truly-running files in
a dedicated set: the pump registers a file at poll-time atomically with
the queue pop, and deleteFile/renameFile either find it running (refuse)
or cancel its queue entry under the same lock (proceed). Batch progress
shrinks when a queued file is cancelled (batchRemove) so the bar still
ends; row status now distinguishes QUEUED from RUNNING on rescan.
2026-09-15 10:15:28 -05:00
avi
757595aaed Desktop: move folder path + Choose-folder into Settings
Library header keeps only wordmark/Settings plus the auto-transcribe
and Transcribe-all controls. Settings gains a 'Recordings folder'
section (full path + Change folder…) at the top; the empty library now
shows a card that jumps straight to Settings.
2026-09-15 10:15:28 -05:00
avi
957b4d7c77 Desktop: Transcribe-all button with batch progress bar
Toolbar gets 'Transcribe all (N)' when N files still lack a transcript
(includes previously-failed ones — asking again means retry). Queues
them through the existing pump; hidden while a batch runs or when the
library is fully transcribed.

While a batch runs, a card shows 'Transcribing batch: k of N — <current
file>' with a bar that blends the settled-file count with the current
file's own backend-reported fraction. Disconnected mid-batch clears the
bar instead of stranding it.

Pure logic in LibraryQueue (transcribeAllPlan, batchAdvance) + 3 new
tests; app suite 37/37. Verified button renders on live build 07:34.
2026-09-15 07:37:14 -05:00
avi
087e8386f5 Transcription: dwindle model choice to Whisper Base + Whisper Large v3
Registry now lists exactly two models instead of five:
- 'Whisper Base' — names the actual bundled model (was 'Base (default)').
- 'Whisper Large v3' — kept; honest copy on size/speed/RAM.
tiny/small/medium removed from the registry (validate_model_name now
rejects them; existing rows keep their saved model — all current data
is 'base', still valid).
Detail screen shows the model that WILL run by real display name
('Whisper Base (default)') instead of 'Use default (base)'; the picker
lists the two models with the default marked inline.
Backend 78 passed/1 skipped; app 34/34; engine hot-reloaded clean.
2026-09-15 07:24:57 -05:00
avi
7234a62eb5 Debian support: verified build/packaging path + friend install guide
- setup-standalone.sh: falls back to system python3 -m venv + pip when
  uv isn't installed (Debian/Ubuntu path; verified on clean bookworm).
- dist/Dockerfile.debian: debian:bookworm build env for :app:packageDeb
  (needs fakeroot + /usr/share/desktop-directories — both baked in; the
  dir gap made jpackage's postinst fail with exit 3 on minimal systems).
- dist/make-release.sh: clean source tarball from HEAD.
- INSTALL-debian.md: end-to-end install for friends (source tree must
  stay on disk — the app spawns the engine from it via SHONAR_REPO).

Verified on real Debian 12 in Docker: :app:packageDeb builds
shonar-desktop_0.1.0-1_amd64.deb, apt installs it, setup-standalone.sh
provisions the engine (venv+pip path, model fetch, boot smoke test),
and the packaged binary launches under Xvfb.
2026-09-15 07:10:27 -05:00
avi
5859a082d2 Desktop: make Ctrl+-/= zoom actually work
Two compounding bugs:
1. Nodes skipped in a plain recomposition kept their old captured
   density, so changing the zoom state left the tree unscaled — wrap the
   CompositionLocalProvider in key(zoom) to force full-tree recompose.
2. Compose focus routing missed the shortcut depending on which child
   held focus — add an AWT KeyboardFocusManager dispatcher below Compose
   (also accepts Ctrl+Up/Down, numpad +/-, Ctrl+0 reset).

User-confirmed zooming works on the live app.
2026-09-15 06:52:30 -05:00
avi
b45e9ff216 Desktop: replace tiny Swing folder chooser with in-app themed picker
JFileChooser rendered unusably small on this HiDPI/Wayland compositor and
had no resize grip. Replace it with a Compose dialog matching the app's
navy+teal skin: breadcrumb row (Home + path steps), scrollable list of
subfolders as cards (dot-dirs hidden), '..' row, and a confirm button
naming the selected folder. Scales with the app's Ctrl+-/= zoom since it
lives inside the Compose tree. Pure nav logic in FolderPicker with 3 new
tests (34 total green).
2026-09-15 06:41:05 -05:00
avi
3e0e3b50cb Desktop: adopt the Android app's companion design (navy+teal sonar skin)
Theme.kt now copies the Android ui/theme dark scheme verbatim: deep navy
#0B1220 + navy surfaces + teal ping #4FD1C5 — replaces warm-graphite gold.
Headers carry the identity: SonarLogo (same geometry as the Android
launcher's ic_launcher_foreground: center dot, teal upper-right arcs,
teal-dark lower-left) + S.H.O.N.A.R. wordmark + 'Self-hosted notes and
audio recording' tagline. Action buttons use the phone's icon vocabulary
(Filled PlayArrow/Edit/Settings/Stop/Delete). Row 'Open' button removed —
clicking the card already opens the recording.
Verified on-screen: navy+teal render, wordmark + tagline, icons, Open gone.
Ctrl+=/- zoom was already ported (4944fcd) and confirmed present.
2026-09-15 06:26:16 -05:00
avi
0a713879b9 Desktop: progress-while-open, search-within-recording, click-hit-to-playback
- openDetail attaches to a pump-owned file with no mapping yet: mirrors the
  pump's live stage/progress until it leaves in-flight, then picks up the
  mapping/report — opening a mid-transcribe file now shows movement instead
  of an idle Transcribe button.
- Detail screen gets a Play/Stop row and a 'Search this recording' box.
  Searchable lines come from live segments or by parsing the [HH:MM:SS]
  prefixes out of the saved report (round-trip tested); each hit card shows
  its timestamp and plays the audio from that moment on click.
- Playback via mpv (external process; Java can't decode m4a) — exact flag
  set verified on this box against a real recording. Stop on leave-screen,
  on file delete, and at app shutdown; natural end clears the playing
  state. 31/31 tests green.
2026-09-15 05:45:30 -05:00
avi
fb1d0e4880 Desktop: multi-select library rows for bulk delete
Checkbox on every row; selecting any reveals a toolbar (N selected,
Select all/Clear all, Clear, red Delete N). One confirmation dialog names
every file (first 8 + count) before anything goes. deleteFiles() re-checks
in-flight state at confirm time and skips anything the pump grabbed since
selection (named in the notice) instead of yanking a live transcribe.
Selection keys on absolute paths and prunes vanished files on rescan so
the count never lies. Selection logic pure + unit-tested (28/28).
2026-09-15 05:37:56 -05:00
avi
a5aedb2b0a Desktop: delete a recording from the app (library row + detail, with confirm)
Library rows and the detail header get a red Delete action that removes the
audio, its .transcript.md report, and its .shonar.json mapping, then purges
the engine's copy in the background (DELETE ?purge=true). Refuses while a
file is queued/transcribing so the pump can't resurrect it mid-flight; a
failed background purge surfaces as an inline warning instead of blocking
the local delete (stale mappings that 404 are silently fine). Confirmation
dialog on both entry points states exactly what goes. deleteSet() is pure
and unit-tested (25/25 green).
2026-09-15 05:14:20 -05:00
avi
c25d3ab150 Bundle the whisper base model + one-shot standalone provisioning
- models/hf-cache/ vendors Systran/faster-whisper-base (142MB, gitignored);
  engine spawn pins HF_HUB_CACHE there, so transcription works with a cold
  ~/.cache and fully offline
- verified: HOME=/tmp/fakehome HF_HUB_OFFLINE=1 model loads and transcribes
  via the bundled cache alone
- setup-standalone.sh: idempotent fresh-clone provisioning (local venv,
  model copy-or-download, engine boot smoke on :8123) — runs green
- backend/README rewritten for the desktop bundled-lite reality (was
  pointing at a nonexistent root README/docs and a Postgres quickstart)

Kotlin suite still green (gradlew test, 0 failures)
2026-09-14 21:05:41 -05:00
avi
a94609c07b Backend tests run on SQLite by default (78 passed, 1 skipped); no Postgres needed
- conftest: default SHONAR_TEST_DATABASE_URL is a temp SQLite file, matching
  the bundled-lite engine; set the env var to a PG URL to exercise that path
- models.py: add sqlite_where to the two partial unique indexes — without it
  SQLite built a FULL unique index on recording_id (WHERE not carried over),
  wrongly blocking a second export asset per recording
- test_m9: pass UUID objects (not str) to direct ORM inserts/gets; SQLite's
  GUID bind processor rejects strings (asyncpg tolerated them)

Verified: pytest -q = 78 passed, 1 skipped; ruff check clean
2026-09-14 20:59:49 -05:00
avi
adc40201a1 Engine venv is now local to this repo (uv venv + editable install of this backend)
- backend/.venv was a symlink into ~/Projects/Shonar; replaced with a real
  local venv built fully offline from the uv cache (deps pinned to the
  versions the app was tested with: fastapi, uvicorn, sqlalchemy,
  aiosqlite, alembic, faster-whisper 1.2.1, dev extras)
- editable install resolves 'shonar' to THIS backend/ (verified via
  shonar.__file__); PYTHONPATH pin kept as a belt
- verified standalone: with ~/Projects/Shonar renamed away, engine boots,
  self-migrates (alembic head m8models000001), /healthz + /readyz ok, and
  full register -> upload -> finalize -> list -> audio-download roundtrip
  passes (sha256 match)
2026-09-14 20:46:50 -05:00