docs: project documentation — architecture, getting-started, packs, mcp, regtest

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 06:48:38 +02:00 · 2026-06-28 06:48:38 +02:00 · 2baad92089
commit 2baad92089
parent 7f4a71b98d
9 changed files with 1392 additions and 0 deletions
--- a/docs/system-map.md
+++ b/docs/system-map.md
@ -0,0 +1,299 @@
+# aiolabs system map
+
+The orientation doc for someone returning to this stack after a
+break — or arriving fresh. Where everything lives, how the pieces
+fit together, who holds which keys, where to start for common
+tasks.
+
+For omni-the-desktop specifically (Hyprland, themes, mksystem,
+flake structure), see [`ARCHITECTURE.md`](ARCHITECTURE.md). For
+fleet deploy mechanics, see [`deploy-strategy.md`](deploy-strategy.md).
+This doc is the **cross-project** map.
+
+---
+
+## The intent
+
+The aiolabs stack is a self-hosted, Bitcoin/Lightning- and Nostr-
+native ecosystem built around the operational needs of **Château du
+Faune** — a collective farm + artist residency in Ariège, France —
+designed from the start to be adopted by other communities with
+similar shape (co-ops, intentional communities, small-scale
+agriculture, retreat centers).
+
+The principles:
+
+1. **Self-hostable end to end.** No SaaS dependencies in the critical
+   path. Every running service has a corresponding entry in `aiolabs/
+   server-deploy` and can be reproduced on commodity hardware.
+2. **Open protocols over proprietary schemas.** Nostr (NIP-52,
+   NIP-72, NIP-46), iCalendar VTODO, ActivityStreams. Other
+   communities can adopt the same shape; their renderers and ours
+   interoperate by virtue of being on the same wire.
+3. **Internal-network defaults.** The community-organizer
+   capabilities work during WAN outages — `docs.ariege.io`, the
+   Nostr relay, LNbits, and the maubot daemon all colocate on
+   `host1` so the foyer e-ink panel and the Matrix bots keep
+   functioning without the internet.
+4. **Identity is the user's, not the operator's.** Per-user Nostr
+   identity, held by a sidecar nsec bunker, with per-device scoped
+   tokens for every client that signs on the user's behalf. See
+   *Trust boundaries* below.
+
+---
+
+## Component map
+
+```
+                          ───── aiolabs stack ─────
+
+  ┌─────────────────────────────────────────────────────────────────┐
+  │                     DEV BOX (this machine — bohm)               │
+  │                                                                  │
+  │   /etc/nixos (omni) ──── this machine's NixOS + home-manager  │
+  │   ~/dev/   ──────────────  worktree of every project repo       │
+  │                                                                  │
+  └────────────────────────────────────┬─────────────────────────────┘
+                                       │ deploys via
+                                       ▼
+  ┌─────────────────────────────────────────────────────────────────┐
+  │                  FLEET HOSTS (via server-deploy)                 │
+  │                                                                  │
+  │   host1 (Château du Faune)     host2    host4                │
+  │      • Continuwuity            • dev     • staging             │
+  │        (Matrix homeserver)     • lnbits  • webapp demo         │
+  │      • maubot daemon                                              │
+  │      • LNbits + nsecbunkerd                                       │
+  │      • Nostr relay                                                │
+  │      • castle-docs (Quartz)                                       │
+  │      • nginx + sops                                               │
+  │                                                                  │
+  │   host5    host3    host6 (WG hub)                              │
+  │      production-tier hosts                                         │
+  └────────────────┬──────────────────────────────┬─────────────────┘
+                   │                              │
+                   ▼                              ▼
+  ┌────────────────────────────┐    ┌──────────────────────────────┐
+  │   USER-FACING SURFACES     │    │  PHYSICAL DEVICES            │
+  │                            │    │                              │
+  │   webapp (PWA)             │    │  inky-impression (Pi + eink) │
+  │     activities / wallet    │    │     foyer display            │
+  │     forum / market / chat  │    │                              │
+  │                            │    │  bitSpire ATM                │
+  │   Matrix client (Element)  │    │     Nostr-native kiosk       │
+  │                            │    │                              │
+  │   docs.ariege.io           │    │  tufty-badge, InkyImpression │
+  │     (Quartz static site)   │    │  LivestockGuardDogs          │
+  │                            │    │     standalone hardware      │
+  └────────────────────────────┘    └──────────────────────────────┘
+```
+
+### What lives where in `~/dev/`
+
+| Path | Role | Where to read more |
+|---|---|---|
+| `deploy/server-deploy/` | NixOS flake for the fleet — single source of truth for what runs on which host | `flake.nix` + per-host modules |
+| `lnbits/main/`, `lnbits/dev/`, `lnbits/nostr-transport/` | Lightning + Nostr identity backend | `lnbits/main/CLAUDE.md`; per-extension READMEs |
+| `webapp/main/`, `webapp/dev/` | Vue/Vite/Electron PWA — user-facing apps over Nostr | `webapp/main/CLAUDE.md` |
+| `bitspire/bitspire/{main,dev}`, `bitspire/atm-tui/` | bitSpire — KYC-free, Nostr-native Bitcoin ATM frontend (+ atm-tui) | `bitspire/bitspire/CLAUDE.md` |
+| `maubot-plugins/` | Matrix bot plugins: `journal`, `tracker`, `wiki` | `maubot-plugins/CLAUDE.md`; `maubot-plugins/docs/community-organizer-spec.md` |
+| `inky-impression/` | E-ink renderer for the foyer panel | `inky-impression/CLAUDE.md` |
+| `docs/castle-docs/` | Operational wiki (Quartz, Obsidian-flavored markdown) | `docs/castle-docs/CLAUDE.md` |
+| `lnbits-extensions/` | The aiolabs LNbits extension catalog (`extensions.json`) | `~/dev/CLAUDE.md` — see "Forgejo conventions" |
+| `shared/extensions/` | LNbits extension source tree (mounted into dev compose) | Per-extension READMEs |
+| `local/docker/regtest/` | Local Lightning regtest stack for end-to-end testing | `local/README.md` |
+| `refs/` | Curated mirrors of upstream reference codebases (Nostr NIPs, Khatru, nostr-tools, etc.) | `refs/README.md` |
+| `upstream-prs/` | Branches staged for upstream contribution | per-branch context |
+
+### Per-project CLAUDE.md (defer to these for repo-specific work)
+
+The cross-cutting facts live in `~/dev/CLAUDE.md`. Anything more
+specific belongs in the per-project `CLAUDE.md`. Start at the most
+specific one for your task and walk outward only if the answer isn't
+there.
+
+---
+
+## The end-to-end data flow
+
+This is the picture once everything's wired up. Some pieces (per-user
+signing) are in flight — see *Status snapshot* below.
+
+```
+   USER (in a Matrix room)
+       │
+       │  !task fix the south fence #urgent
+       ▼
+   Continuwuity homeserver  ───────  Matrix federation
+       │
+       ▼
+   maubot daemon (tracker plugin)
+       │
+       │  resolve binding: MXID → LNbits user
+       │  resolve signer:  LNbits → bunker URL + scoped token
+       │
+       │  build NIP-52 event (kind 31922, community a-tag)
+       │  NIP-46 RPC to nsecbunkerd ──→ sign as user
+       │
+       ▼
+   Internal Nostr relay (loopback / wss://lnbits.<domain>/nostrrelay)
+       │
+       │  fan out to public relays (relay.ariege.io, …)
+       │
+       ▼
+   PUBLIC NOSTR
+       │
+       ├─── inky-impression subscribes  ──→ renders foyer scene
+       │
+       ├─── webapp subscribes           ──→ shows user's items in PWA
+       │
+       └─── any other subscriber        ──→ mobile push, etc.
+```
+
+The community-organizer protocol that defines these event shapes is at
+`~/dev/maubot-plugins/docs/community-organizer-spec.md`. It's
+runtime-agnostic — other projects can produce the same NIP-52 events
+from any source (CLI, web form, voice assistant) and the renderers
+don't care.
+
+---
+
+## Trust boundaries (where keys live)
+
+Knowing what's compromised when X is compromised is the first thing
+to need when something breaks. The honest picture:
+
+| Key | Where it lives | Compromise impact | Reference |
+|---|---|---|---|
+| **User target keys** (`X_alice`, `X_bob`, …) | `nsecbunkerd` sidecar on the LNbits host, encrypted at rest with a passphrase the operator supplies at boot | Catastrophic for user identity *if* the bunker itself is breached. Bunker has a small attack surface (no web server, no plugins). | `aiolabs/lnbits#9`, `#18` |
+| **Per-device scoped tokens** (tracker, ATM, webapp, …) | The client device that holds them. Tracker holds them in its SQLite. Webapp holds them in IndexedDB/localStorage. ATM holds them in its config. | Bounded by token scope. A compromised tracker can publish NIP-52 events as the user; nothing else. Revoke the token at the bunker side; target key unaffected. | spec §7.3 |
+| **Operator master `M_lnbits`** | `nsecbunkerd` admin key. Used by LNbits to authorize admin RPCs (create-account, issue-token, revoke). | Full identity-pool admin access. Operator is on the hook. | `aiolabs/lnbits#18` |
+| **LNbits agent npub** | LNbits process memory, sops-managed at rest | Allows the holder to call `create_account` / `create_token` / `revoke_user` — but NOT to sign as users directly (no nsec). Bounded compared to holding the master. | `aiolabs/lnbits#18` admin client section |
+| **maubot bot keypair** (`@trackerbot:ariege.io`) | maubot's sops-managed config on host1 | Can publish events under the bot's own pubkey (used as fallback when a user is unbound). No access to user signing. | `server-deploy/modules/services/maubot.nix` |
+| **Server identity (`NOSTR_TRANSPORT_PRIVATE_KEY`)** | LNbits env var, sops-managed | Allows impersonation of the LNbits instance itself on the Nostr transport channel. Migrating to bunker-signed in #18. | `aiolabs/lnbits#4`, `#9` |
+| **Wireguard / SSH host keys** | Per host, NixOS-generated | Lateral movement between fleet hosts. Operator concern. | `server-deploy/hosts/<host>/` |
+
+### What's public vs community-private
+
+| Channel | Audience |
+|---|---|
+| `wss://relay.ariege.io` | Public Nostr — anyone can subscribe. Community-organizer events for outward-facing rooms land here. |
+| `wss://lnbits.ariege.io/nostrrelay/<channel>` | Auth-gated internal relays. `nsecbunker` channel is bunker RPC; other channels carry community-scoped events that shouldn't leave the host. |
+| Matrix rooms | Federated by default. Set rooms to invite-only for community-private discussions; the bot reads room membership as trust boundary. |
+| LNbits API | Per-user auth (NIP-98 or session). Bot agent has its own scoped credentials. |
+| `docs.ariege.io` | Public read; published Quartz site. Castle-docs `private/` directory is git-ignored. |
+
+---
+
+## Cross-repo dependencies
+
+When you change repo A, repo B may need a follow-up to actually
+deploy. The chain:
+
+```
+   ~/dev/<project>/                  changes go here first
+        │
+        ▼
+   `aiolabs/<project>` (forgejo)     push to the canonical remote
+        │
+        ▼
+   `aiolabs/server-deploy` flake.lock  bump the input pointing at <project>
+        │  (`nix flake lock --update-input <project>`)
+        ▼
+   deploy.sh <host>                  rolls the change to the host
+```
+
+Catalogs (LNbits extension catalog, etc.) are the exception — they
+read live from `aiolabs/lnbits-extensions/extensions.json` at install
+time, so no flake.lock bump is needed there. See
+`~/dev/CLAUDE.md` "Forgejo conventions" for the per-repo specifics
+(direct-to-main vs PR, fork versioning scheme, catalog rules).
+
+---
+
+## Status snapshot (2026-05-26)
+
+Not everything in the data-flow diagram is shipped. The current state:
+
+| Capability | Status |
+|---|---|
+| Matrix + Continuwuity homeserver on host1 | ✅ live |
+| `journal` maubot plugin (`@journalbot:ariege.io`) | ✅ live |
+| `tracker` maubot plugin (Phase 1 — Matrix + SQLite, rules classifier) | ✅ in repo, ready to deploy |
+| `wiki` maubot plugin (HTTP search against `docs.ariege.io`) | ✅ in repo, ready to deploy |
+| Community-organizer protocol spec | ✅ at `~/dev/maubot-plugins/docs/community-organizer-spec.md` |
+| LNbits Nostr-transport (HTTP-free API) | ✅ merged (#4) |
+| LNbits signer abstraction + transitional LocalSigner | ✅ Phase 1 of #9 — PR #17 (open, awaiting cascading merge) |
+| **`nsecbunkerd` aiolabs fork functional** | ✅ Fork at `aiolabs/nsecbunkerd@master`; 4 upstream-rot patches applied (`#1`–`#4`, `#8`); native nix flake at commit `711a017`; 4 known upstream issues still open (`#5`/`#6`/`#7`/`#8` echo + multi-arch + getKeys) |
+| **LNbits bunker provisioning primitive (phase 2.1)** | ✅ PR #19 (open, stacked on #17) — `NsecBunkerAdminClient` + `RemoteBunkerSigner`; live `ping` + `create_new_key` round-trip ~30ms |
+| LNbits NIP-46 `sign_event` over bunker (phase 2.2-2.3) | 🚧 blocked on upstream `aiolabs/nsecbunkerd#4`/`#7` stabilizing |
+| `error_code` wire vocabulary (14 codes, ATM ↔ LNbits) | ✅ agreed; lands alongside phase 2.3 — see `~/dev/coordination/log.md` |
+| Extension `account.prvkey` migration to `resolve_signer` | 🚧 pre-cascade prerequisite for PR #17 — umbrella at `aiolabs/lnbits#21`; per-extension issues filed (`nostrmarket#5`, `restaurant#11`, `tasks#3`, `events#23`, spirekeeper TBD) |
+| Tracker Nostr publish bridge (Phase 2a) | ⏸ waiting on bunker phase 2.3 |
+| Per-user signing via bunker (Phase 2c) | ⏸ waiting on bunker phase 2.3 |
+| `inky-impression` Nostr-native scene plugin | ⏸ waiting on tracker publishing |
+| Webapp Nostr identity via NIP-46 | ⏸ waiting on bunker phase 2.3 |
+
+### 2026-05-26 day's shipping
+
+- **Cross-session coordination log** established at `~/dev/coordination/log.md` (protocol in sibling `README.md`). Replaces user-relayed messages between the parallel `lnbits`, `bitspire`, `spirekeeper`, and `alfred` sessions. Append-only, chronological.
+- **LNbits PR #19** opened — operator-IdP signup primitive via `aiolabs/nsecbunkerd`. Stacked under PR #17 awaiting cascading merge.
+- **`aiolabs/nsecbunkerd`** — 4 upstream-rot patches landed on `master` so the fork builds + runs from a clean clone (`06272c8`, `960b939`, `42dbbd7`, `e39eaa6`); 8 issues filed total documenting the upstream gaps.
+- **`aiolabs/lnbits#20`** — outbound `_fanout_payment` defense-in-depth dedup (low priority, symmetric to `aiolabs/bitspire#50`).
+- **`aiolabs/lnbits#21`** — umbrella audit catching 5 extensions still reading `account.prvkey` directly. Each affected ext now has its own tracking issue (or pre-existing for `nostrmarket#5`).
+- **Canonical sat-amount vocabulary** locked across spirekeeper / bitspire / atm-tui: `wire_sats` / `principal_sats` / `fee_sats` / `fee_fraction` (unit fraction, never percentage). See `~/dev/coordination/log.md` 2026-05-26T17:10Z + 18:50Z entries.
+- **`bitspire` #49 + #50** closed (Schnorr-verify on RPC replies + two-tier hash dedup on subscribe_payments callbacks). Symmetric lift on the bunker-client side landed in PR #19 commit `4ebcd959`.
+
+---
+
+## Start-here runbook
+
+| I want to… | Read first |
+|---|---|
+| Add a new aiolabs project | `~/dev/CLAUDE.md` (workspace conventions); pick the repo's own CLAUDE.md once it exists |
+| Deploy a change to a fleet host | `deploy-strategy.md` in this dir; `aiolabs/server-deploy` README |
+| Add a new maubot plugin | `~/dev/maubot-plugins/CLAUDE.md`; the multi-line `@command.passive` footgun in `~/dev/CLAUDE.md` |
+| Modify a verb / event shape | `~/dev/maubot-plugins/docs/community-organizer-spec.md` **first**, then the plugin code |
+| Add a page to the operational wiki | `~/dev/docs/castle-docs/CLAUDE.md`; commit + push triggers Quartz rebuild |
+| Render something on the foyer e-ink panel | `~/dev/inky-impression/` (currently HTTP-screenshot; Nostr scenes are planned) |
+| Fork or bump an LNbits extension | `~/dev/CLAUDE.md` "Extension version-bump procedure" |
+| Add a NixOS option / tool on this dev box | `/etc/nixos/home.nix` or `/etc/nixos/configuration.nix`, then `omni-rebuild` |
+| Where are user nsecs? | `nsecbunkerd` on the LNbits host. See `aiolabs/lnbits#9`, `#18`. **Not in lnbits's DB.** |
+| Where do connection tokens live? | The client device (maubot SQLite, webapp localStorage, ATM config). Revoke at the bunker. |
+
+---
+
+## Glossary
+
+A few terms recur across the stack:
+
+- **IdP** — identity provider. In our stack, LNbits is the IdP for end
+  users; it brokers token issuance against the sidecar bunker.
+- **Bunker** — `nsecbunkerd` (or any NIP-46-compliant sidecar) that
+  holds the actual nsec material. Speaks kind-24133 (signing) and
+  kind-24134 (admin) over a relay.
+- **Target key** — a per-user nsec held inside the bunker. Addressable
+  by its npub; never leaves the bunker.
+- **Operator master `M_lnbits`** — the bunker's admin key. Used to
+  authorize admin RPCs (create-account, issue-token, revoke).
+- **Connection token** — a per-device NIP-46 credential, scoped to
+  specific event kinds, issued by the bunker to a specific client
+  (tracker, webapp, ATM). Compromise leaks only the scope.
+- **Signer abstraction** — the `NostrSigner` ABC in
+  `lnbits/core/signers/` and mirrored in tracker. Four implementations:
+  `BotSigner`, `RemoteBunkerSigner` (steady state), `ClientSideOnly
+  Signer` (sovereignty exit), `LocalSigner` (transitional migration
+  helper).
+- **Community a-tag** — NIP-72 community reference, `34550:<pubkey>:
+  <d-tag>`. The way every community-organizer event scopes itself to
+  a community.
+- **Capture / clarify** — GTD framing. `!add` is capture (frictionless,
+  inbox); the rules classifier (or future LLM) does the clarify step
+  that sorts into a kind.
+- **Universal verb vs community shortcut** — the spec mandates universal
+  verbs (`!add`, `!task`, `!journal`, `!remind`, `!done`, `!list`).
+  Communities configure their own shortcuts (`!buy`, `!steward`, …)
+  per room.
+- **Spec-first** — the working convention for cross-system features:
+  write the protocol doc before the implementation. See community-
+  organizer-spec.md as the worked example.