aiolabs/libra
8
0
Fork 1
Code Issues 36 Pull requests Projects Releases 6 Packages Wiki Activity Actions

Route account writes for the split Fava ledger layout #32

Merged
padreug merged 5 commits from feat/split-ledger into main 2026-06-06 18:03:26 +00:00
Conversation 0 Commits 5 Files changed 5 +191 -35
padreug commented 2026-06-06 17:59:29 +00:00
Owner
Copy link

Companion to aiolabs/server-deploy#4 (Fava module: split /var/lib/fava/ledger.beancount into purpose-specific include files). Adapts libra's two ledger-write paths to the new layout.

Closes #28, closes #29.

Summary

  • fava_client.add_account: new target_file parameter + _infer_target_file helper that routes user-account opens (:User-[0-9a-f]{8}$) to accounts/users.beancount and everything else to accounts/chart.beancount. Falls back to inference when callers don't pass it explicitly.
  • fava_client._resolve_target_file: hot-fix discovered during live testing — fava's /api/source rejects relative paths with HTTP 500 (NonSourceFileError). Resolver prepends the absolute dirname of the root ledger (cached on the singleton after first GET /api/options, guarded by an asyncio.Lock so concurrent first-callers don't double-fetch).
  • Always-append-at-end: the original "find last Open, insert after its metadata" heuristic was a clever optimisation for the monolithic ledger but had a pre-existing prefix-match bug (only matched 'open ' or current-year-dated lines, so 1970-* seed opens were invisible). Post-split each include has one mutation profile and append-only is cleaner; new entries land at end of file.
  • POST /api/v1/admin/accounts (new endpoint): super-user-only; takes {name, currencies, description?}; writes the Open to accounts/chart.beancount and syncs into libra's DB via sync_single_account_from_beancount. Validates the name against the five Beancount top-level prefixes. Backs the upcoming admin chart-edit UI (libra#30).
  • account_sync.sync_single_account_from_beancount: now reads the full user_id from Beancount metadata when present, falling back to the name-derived 8-char prefix. crud.py writes the full user_id into metadata when calling fava.add_account, so the post-sync row matches what get_or_create_user_account queries for — kills three lines of warning churn per user-account creation that went through the UNIQUE-constraint recovery path.
  • Startup race fix: tasks.wait_for_account_sync awaits a new fava_client.wait_for_fava_client() helper backed by an asyncio.Event. Previously the sync task started in libra_start() raced the fire-and-forget _init_fava() coroutine and reliably crashed the first iteration with Fava client not initialized.

Verified on aio-demo (live)

After ./deploy.sh aio-demo with /var/lib/fava wiped, then manual sync of the libra files:

  • User signup (jordan, then nance) → both Opens landed in /var/lib/fava/accounts/users.beancount; accounts/chart.beancount untouched
  • Admin POST /api/v1/admin/accounts (three SmokeTest entries) → landed in accounts/chart.beancount; last one at the very bottom (always-append verified)
  • User expense (!-flagged drill purchase) → landed in transactions.beancount via fava's default-file routing
  • Approve → in-place flip ! → * on the same line; no collateral edits to any other file. Log line confirmed fava reports meta['filename'] as the absolute path to the include file, which /api/source accepts as-is.
  • Income entry + manual EUR edit + partial settlement → bean-check PASS across all three entries
  • Read-only boundary: sudo -u fava bash -c 'echo x >> options.beancount' → Permission denied; same on transactions.beancount succeeds. Server-deploy@29e9b09 chmod'd ledger/options/commodities to 0440.
  • Startup race fix: clean restart with no "Fava client not initialized" error; sync runs after init event fires

⚠️ Catalog-bump rollout — cfaun upgrade must wait

Once this is merged and the catalog version is bumped, ANY host pointing at our manifest can pull the new libra via the lnbits admin UI's "Upgrade" button. Do not click Upgrade on cfaun until aiolabs/server-deploy#6 (cfaun ledger migration) has run. Running this libra against cfaun's still-monolithic /var/lib/fava/ledger.beancount would misroute every write (Opens to non-existent accounts/{chart,users}.beancount, transactions to a non-existent default-file route).

aio-demo is already on the split layout (deployed earlier in this session) so its Upgrade is safe whenever.

Companion to `aiolabs/server-deploy#4` (Fava module: split `/var/lib/fava/ledger.beancount` into purpose-specific include files). Adapts libra's two ledger-write paths to the new layout. Closes #28, closes #29. ## Summary - **`fava_client.add_account`**: new `target_file` parameter + `_infer_target_file` helper that routes user-account opens (`:User-[0-9a-f]{8}$`) to `accounts/users.beancount` and everything else to `accounts/chart.beancount`. Falls back to inference when callers don't pass it explicitly. - **`fava_client._resolve_target_file`**: hot-fix discovered during live testing — fava's `/api/source` rejects relative paths with HTTP 500 (`NonSourceFileError`). Resolver prepends the absolute dirname of the root ledger (cached on the singleton after first `GET /api/options`, guarded by an `asyncio.Lock` so concurrent first-callers don't double-fetch). - **Always-append-at-end**: the original "find last Open, insert after its metadata" heuristic was a clever optimisation for the monolithic ledger but had a pre-existing prefix-match bug (only matched `'open '` or current-year-dated lines, so 1970-* seed opens were invisible). Post-split each include has one mutation profile and append-only is cleaner; new entries land at end of file. - **`POST /api/v1/admin/accounts`** (new endpoint): super-user-only; takes `{name, currencies, description?}`; writes the Open to `accounts/chart.beancount` and syncs into libra's DB via `sync_single_account_from_beancount`. Validates the name against the five Beancount top-level prefixes. Backs the upcoming admin chart-edit UI (`libra#30`). - **`account_sync.sync_single_account_from_beancount`**: now reads the full user_id from Beancount metadata when present, falling back to the name-derived 8-char prefix. crud.py writes the full user_id into metadata when calling `fava.add_account`, so the post-sync row matches what `get_or_create_user_account` queries for — kills three lines of warning churn per user-account creation that went through the UNIQUE-constraint recovery path. - **Startup race fix**: `tasks.wait_for_account_sync` awaits a new `fava_client.wait_for_fava_client()` helper backed by an `asyncio.Event`. Previously the sync task started in `libra_start()` raced the fire-and-forget `_init_fava()` coroutine and reliably crashed the first iteration with `Fava client not initialized`. ## Verified on aio-demo (live) After `./deploy.sh aio-demo` with `/var/lib/fava` wiped, then manual sync of the libra files: - [x] User signup (jordan, then nance) → both Opens landed in `/var/lib/fava/accounts/users.beancount`; `accounts/chart.beancount` untouched - [x] Admin `POST /api/v1/admin/accounts` (three SmokeTest entries) → landed in `accounts/chart.beancount`; last one at the very bottom (always-append verified) - [x] User expense (`!`-flagged drill purchase) → landed in `transactions.beancount` via fava's `default-file` routing - [x] Approve → in-place flip `!` → `*` on the same line; no collateral edits to any other file. Log line confirmed fava reports `meta['filename']` as the absolute path to the include file, which `/api/source` accepts as-is. - [x] Income entry + manual EUR edit + partial settlement → `bean-check` PASS across all three entries - [x] Read-only boundary: `sudo -u fava bash -c 'echo x >> options.beancount'` → `Permission denied`; same on `transactions.beancount` succeeds. Server-deploy@29e9b09 chmod'd ledger/options/commodities to 0440. - [x] Startup race fix: clean restart with no "Fava client not initialized" error; sync runs after init event fires ## ⚠️ Catalog-bump rollout — cfaun upgrade must wait Once this is merged and the catalog version is bumped, ANY host pointing at our manifest can pull the new libra via the lnbits admin UI's "Upgrade" button. **Do not click Upgrade on cfaun until `aiolabs/server-deploy#6` (cfaun ledger migration) has run.** Running this libra against cfaun's still-monolithic `/var/lib/fava/ledger.beancount` would misroute every write (Opens to non-existent `accounts/{chart,users}.beancount`, transactions to a non-existent default-file route). aio-demo is already on the split layout (deployed earlier in this session) so its Upgrade is safe whenever.
The Fava-backed ledger is being split into purpose-specific files (see
aiolabs/server-deploy#4): accounts/chart.beancount for static + admin-managed
opens, accounts/users.beancount for libra-appended per-user opens.

Add a `target_file` parameter to `add_account` that defaults to inference
from the account name (`:User-[0-9a-f]{8}$` -> users.beancount, otherwise
chart.beancount). Drop the now-redundant `GET /api/options` call that was
only used to discover the root file path. Callers that need explicit
control (e.g. the upcoming admin chart-edit endpoint) can pass
`target_file=` directly.

The retry loop, write lock, and insertion-point search are unchanged --
each included file is a self-contained source the existing logic operates
on cleanly.

Refs: aiolabs/libra#28
Companion to the fava ledger split (aiolabs/server-deploy#4). Super-user
endpoint that adds a new Open directive to accounts/chart.beancount via
fava_client.add_account (explicit target_file), then mirrors the account
into Libra's DB via sync_single_account_from_beancount so permissions can
be granted on it.

Validates the account name against the five Beancount top-level prefixes
(Assets:/Liabilities:/Equity:/Income:/Expenses:) and returns 400 on a bad
prefix.

Per-user accounts (matching :User-xxxxxxxx) keep their existing code path
via crud.get_or_create_user_account, which inherits the inferred target_file
(accounts/users.beancount) from the add_account default.

Backend only -- the LNbits admin UI on top is tracked separately as
aiolabs/libra#30.

Refs: aiolabs/libra#29
Fava's /api/source endpoint rejects relative paths with HTTP 500
(NonSourceFileError: "Trying to read a non-source file at '...'"). The
include-aware `_infer_target_file` helper returns relative paths
(e.g. "accounts/users.beancount"), so add a `_resolve_target_file`
hook that prepends the ledger root directory.

The dirname is derived from a one-time GET /api/options and cached on
the FavaClient instance (which is a module-level singleton), guarded by
an asyncio.Lock so concurrent first-callers don't double-fetch.

Absolute paths pass through unchanged, so the admin endpoint that
explicitly passes target_file="accounts/chart.beancount" works the same
as one that passes "/var/lib/fava/accounts/chart.beancount".

Verified against aio-demo's live fava: relative paths now produce
HTTP 200 reads on options.beancount, accounts/chart.beancount,
accounts/users.beancount, and transactions.beancount.

Refs: aiolabs/libra#28
Three small fixes shaken out by live testing on aio-demo:

1. fava_client.add_account: when the target file has no Open directives
   yet (e.g. the empty accounts/users.beancount seed), append at end of
   file instead of inserting at index 0. Keeps the seed header comments
   at the top where they belong.

2. account_sync.sync_single_account_from_beancount: read the full user_id
   from Beancount metadata when present, fall back to the name-derived
   8-char prefix otherwise. crud.get_or_create_user_account writes the
   full 32-char user_id into Beancount metadata when creating per-user
   accounts; the sync function was only looking at the account name and
   returning the prefix, so the post-sync `WHERE user_id=:user_id` query
   in crud.py missed the row and fell through the UNIQUE-constraint
   recovery path. Three lines of warning noise per user-account creation.

3. tasks.wait_for_account_sync: await `wait_for_fava_client()` (new
   helper backed by an asyncio.Event in fava_client.py) before the first
   sync iteration. Previously the sync task started in libra_start()
   raced the fire-and-forget `_init_fava()` coroutine and reliably
   crashed the first run with "Fava client not initialized".

Refs: aiolabs/libra#28
The original "find last Open directive, insert after its metadata" logic
was a clever optimisation for the monolithic ledger where opens, txns,
and assertions all lived in one file -- you wanted new opens grouped with
existing opens, not appended after a long transaction tail.

Post-split, each include file has one mutation profile:
  - accounts/chart.beancount: only Open directives
  - accounts/users.beancount: only Open directives
  - transactions.beancount:   only Transactions

There is no longer a content shape that benefits from mid-file insertion;
the existing heuristic also had a pre-existing bug where it only matched
'open ' OR '{current_year}-' as line prefixes, so 1970-* seed opens were
invisible and the search "stuck" to the first current-year line in the
file (which on aio-demo ended up being the wrong place).

Drop the search; always append. Simpler, chronological, append-only
friendly.

Refs: aiolabs/libra#28
padreug deleted branch feat/split-ledger 2026-06-06 18:03:26 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
aiolabs/libra!32
No description provided.