diff --git a/modules/dev-env/docs/lnbits-extension-dev.md b/modules/dev-env/docs/lnbits-extension-dev.md new file mode 100644 index 0000000..3f7e27c --- /dev/null +++ b/modules/dev-env/docs/lnbits-extension-dev.md @@ -0,0 +1,143 @@ +# lnbits extension development + +Reference for building and maintaining LNbits extensions — the parts +that catch first-time extension authors and the patterns worth +adopting once you maintain a fork-modified extension. + +## Auth decorators + +Easy to confuse. Different scopes: + +| Decorator | Auth scope | Returns | Use for | +|---|---|---|---| +| `require_invoice_key` | Wallet invoice key — read access | `Wallet` | Read endpoints (balance, payment history) callable with a user's invoice key | +| `require_admin_key` | Wallet admin key — write to **own wallet only** | `Wallet` | Endpoints that create payments / modify wallet state | +| `check_admin` | **LNbits instance admin** (super_user + `lnbits_admin_users`), Bearer token | `Account` | Cross-user / admin-only operations | +| `check_super_user` | LNbits super user only, Bearer token | `Account` | Operations restricted to the single super_user | + +**`require_admin_key` ≠ `check_admin`.** This is the most common +misuse. `require_admin_key` is a *wallet*-level write key — every +user has one for each of their own wallets. `check_admin` is +*instance* admin access. Endpoints that operate on other users' data +or change global settings need `check_admin`, not `require_admin_key`. + +## Testing + +Use **FakeWallet** +(`LNBITS_BACKEND_WALLET_CLASS=FakeWallet`) for testing extension CRUD, +API endpoints, and UI changes. Spin up the full regtest stack only +when end-to-end Lightning payment behavior is actually under test. + +**Why:** regtest takes time to start (docker build + multiple LND/CLN +containers), uses real resources, and adds no fidelity for non-payment +flows. FakeWallet makes payments succeed instantly with no network — +ideal for testing the surrounding logic. + +If you ship a `dev` CLI (see lnbits-sensei's pattern), keep +`--fakewallet` as the default mode for this reason. + +## Fork-migrations pattern (`migrations_fork.py`) + +Long-lived forks need to add schema columns and tables on top of +upstream's. The naive approach — append fork-only migration functions +to `migrations.py` — guarantees merge conflicts on every upstream +rebase. The pattern below sidesteps that by keeping `migrations.py` +byte-identical to upstream and putting fork deltas in a sibling file. + +> **Note:** this pattern requires a patch to LNbits core's +> `migrate_extension_database()` that loads `migrations_fork.py` under +> a `_fork` key in the `dbversions` table. As of 2026 that +> patch is fork-internal (an upstream PR is the natural follow-up). +> If you maintain a fork that ships this patch, the rest of this +> section is the user-facing pattern. + +### Architecture facts + +- `dbversions` lives in the **core LNbits DB** (`database.sqlite3`), + not per-extension. Schema: `(db TEXT PRIMARY KEY, version INT)`. + `update_migration_version` is an INSERT-OR-UPDATE, so a new + `_fork` row appears on first run with no schema migration + needed core-side. +- Extension data tables live in `ext_.sqlite3` (SQLite) or a + Postgres schema named after ``. Created lazily on first + `Database.connect()` via `ATTACH` (SQLite) or `CREATE SCHEMA` + (Postgres). +- The version-bump connection is routed based on `db.schema`: `None` + means a core migration (same connection), set means an extension + (opens a fresh `core_db.connect()` to write `dbversions`). See + `lnbits/core/helpers.py:run_migration`. +- **No cross-DB atomicity.** Extension migration writes commit to + `ext_.sqlite3`; the `dbversions` upsert commits to + `database.sqlite3`. If the extension write succeeds and the + dbversions write fails, the migration is orphaned — re-runs on + next startup. **Every fork migration MUST be idempotent** (use + an `_alter_add_column_safe` wrapper that swallows + duplicate-column errors, `CREATE TABLE IF NOT EXISTS`, etc.). + Self-healing covers the orphan case. + +### Squash recipe (adopting the pattern on an existing fork) + +If your fork has accumulated fork-only migrations interleaved into +`migrations.py`: + +1. **Restore `migrations.py` to upstream-byte-identical content.** + Drop all fork-only `m{NNN}_*` functions and any helper added only + for them. +2. **Create `migrations_fork.py`** with a SINGLE + `m001___schema` function that idempotently applies + every fork-only schema delta the old migrations used to do. One + readable file forever. +3. The squash uses `_alter_add_column_safe` per ALTER and + `CREATE TABLE IF NOT EXISTS` per table — no-ops cleanly on + installs that already ran the old fork migrations. + +### One-time fix on existing installs adopting the pattern + +Installs that previously ran the old fork migrations have their +`dbversions['']` row ahead of upstream (e.g. `events|11` after +fork-only `m007`–`m011`). After moving to `migrations_fork`, the +next upstream rebase that adds e.g. `m007_add_allow_fiat` would +compare `7 > 11 → false` and **silently skip** the new upstream +migration. + +**Reset the row to match upstream's actual migration count** before +the rebase lands: + +```sql +-- Run against the CORE DB (database.sqlite3), not the extension DB. +UPDATE dbversions SET version = WHERE db = ''; +``` + +For containerized dev where the file is root-owned inside the +container: + +```bash +docker compose exec lnbits python3 -c "import sqlite3; \ + c = sqlite3.connect('database.sqlite3'); \ + c.execute(\"UPDATE dbversions SET version = WHERE db = ''\"); \ + c.commit()" +``` + +### Upstream-overlap scenario at rebase time + +If upstream eventually adds a schema change you already carry in +`migrations_fork.py` (e.g. they add the same `ALTER TABLE … ADD COLUMN +bar` you shipped), the next rebase creates a problem: fresh installs +work (your `_alter_add_column_safe` guards swallow the dup), but +**existing installs crash** when upstream's now-redundant migration +runs without an idempotency guard (upstream rarely uses them). + +Mitigation at rebase time: + +1. **Prune the redundant block from `migrations_fork.py`** so future + fresh installs get the column from upstream's migration. +2. **Pre-deploy `dbversions` surgery on every affected install**: + `UPDATE dbversions SET version = WHERE db = ''` + so the loader skips the now-overlapping upstream migration. + +Do both — (1) keeps future fresh installs clean, (2) keeps existing +installs alive through the deploy. + +**Don't** patch upstream's migration with idempotency guards in your +fork. That breaks the "migrations.py == upstream byte-identical" +property and reintroduces every-rebase conflicts. diff --git a/modules/dev-env/docs/lnbits-frontend-gotchas.md b/modules/dev-env/docs/lnbits-frontend-gotchas.md new file mode 100644 index 0000000..cbd1b7b --- /dev/null +++ b/modules/dev-env/docs/lnbits-frontend-gotchas.md @@ -0,0 +1,105 @@ +# lnbits frontend gotchas + +LNbits ships its UI with **Vue 3 + Quasar 2 as UMD globals** — no +build step, plain Jinja templates with per-page JS. This applies +across lnbits core (`lnbits/templates/*.html`), every extension +(`/templates/`), and every fork that doesn't restructure the +frontend stack. The UMD load model has several traps that don't +manifest under the build-step model most Vue tutorials assume. + +## No self-closing tags + +Per [Quasar's UMD usage rules](https://quasar.dev/start/umd/#usage), +components must use the explicit-close form: + +```html + + + + + + + +``` + +**Why:** UMD-loaded templates are parsed by the browser's HTML parser, +not Vue's compiler. The HTML parser doesn't honor self-close on +non-void elements (per the HTML spec). The close tag gets implied at +the wrong place, nesting breaks silently, and subsequent siblings end +up nested inside the prior component. + +Self-closing is fine in `.vue` SFCs (the build step rewrites them +before the browser sees anything), so if you copy a snippet from a +Vue SFC repo into an LNbits template, **expand all self-closing tags +before saving**. + +## CSS specificity trap + +LNbits applies its own theme overrides on Quasar's typography +utilities (`.text-caption`, `.text-grey-*`, etc.) with `!important`. +Class-based CSS rules in an extension page — *even with `!important`* — +lose this fight unless your selector is strictly more specific than +the upstream rule. + +**Rule:** for per-element typography/color overrides on LNbits pages, +reach for Vue `:style` bindings (or static `style="..."` attrs), not +` + + +… +… +``` + +Background/border tweaks at card-level via class are fine — the trap +is specifically the typography utilities (`text-*`) and Quasar's +color utilities. + +## Cache busting + +Static assets are served with `?v={server_startup_time}` appended +(see `static_url_for` in `lnbits/helpers.py`). Consequences: + +- **Bumping JS requires a server restart.** Reloading the browser + doesn't help if `?v=` hasn't changed — the browser keeps serving + the cached file. +- **Jinja templates re-render on every request** (the `?v=` is only + on static assets). No restart needed for template edits — just + refresh. + +If a browser keeps serving stale JS after a restart, hard-refresh +(`Ctrl+Shift+R`) to bypass the HTTP cache. + +## Dark-mode color discipline + +LNbits's dark theme inverts text colors on most surfaces but **not** +on `bg-{color}-1` pale-background utilities. Result: a `bg-red-1` +without an explicit text color renders white-on-cream under dark +theme — basically invisible. + +**Rule:** pair every pale-background utility with an explicit dark +text class: + +```html + +
Warning
+ + +
Warning
+``` + +Same for `bg-green-1`, `bg-blue-1`, `bg-amber-1`, etc. The `text-grey-9` +choice is the safe default; pick a darker shade if you want stronger +contrast. + +## When in doubt + +Test under both light and dark themes (Quasar's theme toggle is at +the top of every LNbits page once you're logged in). Most of the +above gotchas are silent under one theme and obvious under the other +— don't ship UI without flipping the toggle at least once. diff --git a/modules/dev-env/docs/lnbits-upstream-flow.md b/modules/dev-env/docs/lnbits-upstream-flow.md new file mode 100644 index 0000000..3284f69 --- /dev/null +++ b/modules/dev-env/docs/lnbits-upstream-flow.md @@ -0,0 +1,128 @@ +# `lnbits/lnbits` — Upstream Development Flow + +Reference for [`github.com/lnbits/lnbits`](https://github.com/lnbits/lnbits). +Verified against git history on 2026-05-18; bump the date when you +re-verify, and patch this doc if the model has drifted. + +## Branch model + +Two long-lived branches: + +| Branch | Role | How it moves | +|---|---|---| +| `dev` | Integration / staging | One squash-merge commit per PR. Linear. | +| `main` | Release | A non-fast-forward merge of `dev` at each release. | + +`main` never receives feature PRs directly. **Every change reaches +`main` through `dev`.** + +## PR flow → `dev` + +1. Contributor opens a PR targeting `dev` (not `main`). +2. Maintainer squash-merges via the GitHub UI. +3. The resulting commit on `dev` has: + - exactly one parent (linear history), + - subject ending in `(#NNNN)` — the squash-merge signature, + - lowercase conventional-commit prefix: `feat:`, `fix:`, `chore:`, + `chore(deps):`, `docs:`, `ci:`, `test:`, `refactor:`. +4. CI runs on the PR; nothing else is required between merge and the + commit appearing on `dev`. + +Example chain on `dev`: + +``` +c9c68bd8 Fix: Use default reaction on bootstrap (#3965) +810a1372 fix: tighten agents file (#3966) +36d696b2 Fix: wrong use of `in` operator (#3960) +8b426efa test: add pyinstrument profiler (#3955) +``` + +Each is a single squashed commit. No merge commits inside `dev`. + +## Release flow → `main` + +When `dev` is ready to ship: + +1. A release-candidate version bump lands on `dev` as a normal PR: + `chore: update to version vX.Y.Z-rcN (#NNNN)`. +2. Validation happens against the RC. +3. A final version-bump PR lands on `dev`: + `chore: update to version vX.Y.Z (#NNNN)`. +4. A maintainer runs the release merge locally: + + ```bash + git checkout main + git pull --ff-only origin main + git merge dev # true (non-FF) merge, default message + git push origin main + git tag vX.Y.Z # tag the merge commit or the bump commit + git push origin vX.Y.Z + ``` + +5. The resulting merge commit on `main` has: + - **two parents** (prior `main` tip + `dev` tip), + - subject exactly `Merge branch 'dev'` (git default when on `main`), + - **not** authored via the GitHub PR-merge UI (that would produce + `Merge pull request #N from …`). + +Because `main` typically carries a stray release-bump commit that isn't +on `dev`, the histories have diverged and git is forced into a true +merge. `--no-ff` is not needed for that reason. + +## Reading the history + +```bash +# Release log (one entry per release): +git log main --first-parent --oneline + +# Full changelog leading into the next release: +git log dev --oneline + +# Verify a merge is a true non-FF merge: +git log -1 --format='%P' # two parent hashes = true merge +``` + +## Diagram + +``` + (PRs squash-merged one at a time) + │ + ▼ +dev: ── A ── B ── C ── D ── E ── F (rc1) ── G ── H (v1.5.4) + ╲ + ╲ (true merge) + ╲ +main: ────────── prev release ─────────────── X ─────────── M + ▲ + │ + Merge branch 'dev' + tag: v1.5.4 +``` + +## Implications for contributors + +- **Base PRs on `dev`.** PRs against `main` will not be accepted. +- **Use lowercase conventional-commit titles.** Verified stable across + the last 25+ merged PRs. +- **Don't expect `main` to move between releases.** It only advances + when a maintainer cuts a release merge. +- **Tagging is on `main`.** Consumers pinning to a tag get the + released state, never a `dev` snapshot. + +## Adapting this in your own fork + +If you maintain a long-lived fork of `lnbits/lnbits` for production use, +you'll likely want to: + +1. Mirror upstream's `dev` / `main` split — easier to track upstream + merges back into your fork. +2. Adopt a version-suffix convention that surfaces fork identity in + tags and the packaged `pyproject.toml` version, e.g. + `v-.`. This makes it unambiguous in logs and + deployed-package metadata that you're running fork-modified code. +3. Decide whether you also need pre-release channels (`-rcN`, `-devN`) + on top of upstream's; useful if your fork ships to staging hosts + before promotion. + +Specifics depend entirely on your team's needs — this scaffold +deliberately doesn't prescribe a fork-versioning scheme. diff --git a/modules/dev-env/docs/lnbits-workspace-notes.md b/modules/dev-env/docs/lnbits-workspace-notes.md new file mode 100644 index 0000000..104af1b --- /dev/null +++ b/modules/dev-env/docs/lnbits-workspace-notes.md @@ -0,0 +1,243 @@ +# lnbits workspace notes + +Practical reference for day-to-day work in an lnbits dev environment. +Collected gotchas, conventions, and design constraints that have +repeatedly surprised people. Not a tutorial — assumes you're already +running lnbits and contributing or extending it. + +## Pick a non-default port for your local dev server + +LNbits defaults to `:5000`. That collides with macOS AirPlay Receiver, +common docker-compose stacks, and other dev tools. Pick something +unambiguous (many forks settle on `:5001`) and stick to it across: + +- `settings.nix` → `lnbits.port` +- any MCP server config you wire against your dev instance +- bookmarks, env files, CI configs + +Once you wire it in three places, switching the port retroactively is +mostly find-and-replace pain. Decide early. + +## LNBITS_SRC and docker-compose build context + +If you run a docker-compose dev stack (regtest or otherwise) that +builds lnbits from a local checkout, the `Dockerfile` typically reads +from `${LNBITS_SRC:-/some/default}`. + +The trap: **commits to your day-to-day worktree don't reach the dev +image if `LNBITS_SRC` is currently pointed elsewhere** (e.g. at a +feature-branch worktree you were testing). Even +`docker compose build --no-cache` happily rebuilds from the wrong +checkout. + +Sanity-check before assuming a rebuild picked up your change: + +```sh +docker compose config | grep -A2 lnbits +# look for the resolved `context:` path +``` + +If it's pointing somewhere stale, either cherry-pick your commit onto +the active build branch or flip `LNBITS_SRC` and rebuild. + +## Extension folder is the install target — "Upgrade" wipes forks + +If your dev setup mounts an extension checkout directly into the lnbits +container (e.g. `~/dev/shared/extensions/` → `/shared`, with +`LNBITS_EXTENSIONS_PATH=/shared`), **the extension git checkout IS the +installed extension**. There's no separate copy. + +Consequence: clicking **"Upgrade"** in the LNbits UI on an extension +that's mounted from a fork checkout will: + +1. Download the catalog tarball. +2. Extract it directly over the mounted directory. +3. **Wipe `.git`**, replace every file with the catalog version, and + silently discard any local changes / fork patches. + +If this happens, recover with `git clone ` over +the wiped directory. Mitigations: + +- Set an extension version in your fork that always sorts above the + catalog's so "Upgrade" never thinks the catalog is newer. +- Or don't expose the catalog upgrade UI to a workflow that has + mounted-fork extensions. + +## Settings precedence: `.env` seeds the DB on first boot, then the DB wins + +This one trips up most people who deploy lnbits declaratively (via NixOS, +docker-compose, ansible, …). Verified 2026-05-24 against upstream +`lnbits/main`. + +LNbits has two sources of truth for settings depending on lifecycle. +**On boot, when `lnbits_admin_ui=True`** (the default): + +1. Read DB row via `get_super_settings()` + (`lnbits/core/services/users.py:236`). +2. If the DB row is empty → seed it from `.env` via + `init_admin_settings()`. First-boot only. +3. `update_cached_settings(settings_db.dict())` overwrites the + in-memory `Settings` with the DB row. **`.env` values loaded by + Pydantic at startup are clobbered.** + +**Practical consequence:** once an instance has booted once, editing +`.env` and restarting **changes nothing** for editable fields. They can +only be changed via: + +- The Admin UI (`PUT /api/v1/settings`, gated by `check_admin`), or +- Clearing the relevant rows in the `system_settings` table in the + core DB. + +**Exceptions where `.env` still wins on every boot:** + +- `super_user` — env overrides DB explicitly in `users.py`. +- `lnbits_admin_ui=False` — the whole DB-load block is skipped; env + stays authoritative (there's no Admin UI to populate the DB anyway). +- All `ReadOnlySettings` fields (defined in `lnbits/settings.py` — + `EnvSettings` / `PersistenceSettings` / `SuperUserSettings` / + `ExtensionsInstallSettings`). Concretely: + + - `host`, `port` + - `lnbits_path`, `lnbits_data_folder`, `lnbits_extensions_path` + - `lnbits_database_url` + - `auth_secret_key`, `first_install_token` + - `lnbits_title` (the API title — NOT `lnbits_site_title`, which is + editable) + - `lnbits_admin_ui` itself + - `lnbits_allowed_funding_sources` (the *list of which sources can be + enabled* — the per-source credentials live in `FundingSourcesSettings` + → `EditableSettings` and ARE DB-frozen) + + `update_cached_settings` skips any key in `readonly_variables`, so + env-loaded values for these fields survive the DB-load overwrite. + +**Editable** settings — site title/tagline, theme, watchdog thresholds, +fee defaults, rate limits, all per-funding-source credentials, the +whole Admin UI form — get DB-frozen after first boot. + +### `LNBITS_FIRST_INSTALL_TOKEN` rotation ≠ "reset settings to env" + +Rotating the first-install token creates a new super_user account with +a fresh random UUID and re-enables the `/first_install` endpoint. It is +an **escape hatch for a locked-out admin** to re-claim the instance +with a new username/password. It does NOT refresh any settings values +from `.env` — by the time it runs, the DB row has already overwritten +cached `settings`, and `init_admin_settings()` then upserts those same +DB values back, so no env values flow through. + +### Deploy-side implication + +If you manage lnbits config declaratively (NixOS module, ansible role, +docker-compose env file), **editable env vars only take effect on a +fresh install** (empty `settings` table). For an existing deployment, +changing them in your declarative source and redeploying won't change +runtime behavior — you have to edit through the Admin UI OR clear the +relevant rows in `system_settings`. + +The "set it in nix, redeploy, done" mental model only works for +`ReadOnlySettings` fields. Anything an admin can edit in the UI is +DB-frozen post-first-boot. Plan your deploy story accordingly: + +- For `ReadOnlySettings` (host, port, paths, secret_key, …): + declarative source is authoritative on every boot. Reproducible. +- For `EditableSettings` (site title, fees, funding-source creds, …): + declarative source is a *seed*, not authoritative. To re-seed an + existing instance, you must explicitly clear the DB row. + +## Nostr key handling — don't persist plaintext nsecs + +As of 2026, LNbits's upstream account model server-generates a Nostr +private key (`accounts.prvkey`) for every new account and stores it +**plaintext in the DB**. The key is returned to clients over HTTPS via +auth endpoints. A DB snapshot, backup leak, or curious operator is a +wholesale identity compromise for every user. + +If your fork or extensions touch user Nostr keys, design around this. +A defensible shape: + +- A `NostrSigner` abstraction with at least three implementations: + - `LocalSigner` — server-side key, but envelope-encrypted at rest. + User unlocks per session. + - `RemoteBunkerSigner` — NIP-46 connection to a bunker the user + runs themselves. Server never sees the private key. + - `ClientSideOnlySigner` — the server only knows the user's + public key. Signing happens entirely in the browser / NIP-07 + extension / hardware signer. +- Optionally a `DelegatedSigner` via NIP-26 for accounts that want + a server-side signer for low-value operations while keeping + high-value ops client-side. + +Also audit every server-side Nostr key your extensions persist +(merchant signing keys, notification keys, transport keys, …) and +either encrypt at rest or document why ephemeral is fine. + +The general principle: **prefer "don't store the key" when possible.** +Let the user's signer (browser extension, hardware device, NIP-46 +bunker) do the signing. + +## CLINK is Lightning.Pub-specific, not LNbits + +Shocknet's [CLINK protocol](https://github.com/shocknet/CLINK) — the +Nostr-native payment flow with event kinds 21001 (offer/noffer), +21002 (debit/ndebit request), 21003 (debit response) — lives inside +`Lightning.Pub`, not LNbits. LNbits as of 2026 has zero CLINK +knowledge. + +If you're thinking about `ndebit` / `noffer` semantics in an lnbits +context: + +- They're Lightning.Pub primitives. LNbits doesn't implement them. +- Building "equivalent outcomes" in LNbits typically goes through + LNURL + payment-hash subscriptions + opaque `extra` payloads + rather than CLINK-style events. Different primitives, similar + effective semantics. +- If you genuinely need CLINK in lnbits, it's a new transport + module, not a tweak to existing code paths. NIP-44 plumbing in + the nostr-transport surface is the natural building block. + +Don't conflate the two — keep CLINK semantics on the Lightning.Pub +side of any cross-system design. + +## Fork versioning — surface your fork in the version string + +If you maintain a long-lived fork (especially one running in +production), the released version string should make it obvious that +fork-modified code is what's running. Consumers of the package, log +readers, and bug reporters all benefit. + +A common pattern: + +- **Git tag:** `v-.` + e.g. `v1.5.4-aio.1`, `v1.5.4-aio.2`, … +- **`pyproject.toml` `version`:** `+.` + e.g. `1.5.4+aio.1`. + +The two spellings differ because PEP 440 doesn't accept +hyphen-with-arbitrary-suffix as a valid package version, but the local- +version form (`+.`) is valid. Extensions don't have the PEP 440 +constraint (their `config.json` is freeform), so the tag form works +everywhere except `pyproject.toml`. + +Decide your own tag string and stick to it. The key property is that +`importlib.metadata.version("lnbits")` and the LNbits UI footer make +it visible to anyone looking. + +For the upstream lnbits branch / release model these tags sit on top +of, see [lnbits-upstream-flow.md](lnbits-upstream-flow.md). + +## Reading the codebase fast + +For mass-grep and reference reading, point a tools-friendly mirror of +the lnbits source at a known location (see your `~/dev/refs/` setup if +you use one). Key paths once you have a checkout: + +- `lnbits/core/services/` — core money flows. Start here for invoice + / payment / wallet logic. +- `lnbits/wallets/` — backend implementations. One file per + `LNBITS_BACKEND_WALLET_CLASS` value. +- `lnbits/extensions/` — first-party extensions. Third-party + extensions follow the same shape (`config.json`, `views.py`, + `migrations.py`, etc.). +- `git log --first-parent dev --oneline` — see the squash-merge + sequence into `dev` (see lnbits-upstream-flow.md for why + `--first-parent` matters here). diff --git a/modules/dev-env/docs/remotes.md b/modules/dev-env/docs/remotes.md new file mode 100644 index 0000000..55cde47 --- /dev/null +++ b/modules/dev-env/docs/remotes.md @@ -0,0 +1,80 @@ +# Remote topology + +`lnbits-sensei.git.remotes` (declared in `modules/git/remotes.nix`) +abstracts how the local LNbits checkout is wired to its git remotes. +Three patterns cover almost every workflow. + +## 1. Upstream-only + +You read upstream, never push. Good for a read-only dev box, a CI +runner, or initial exploration before you've decided to contribute. + +```nix +lnbits-sensei.git.remotes = { + upstream = "https://github.com/lnbits/lnbits"; + fork = null; + extras = [ ]; +}; +``` + +What you get: + +- `upstream` remote pointing at canonical lnbits/lnbits. +- No `fork` remote — the bootstrap script skips it when `fork` is null. +- No extras. + +## 2. GitHub fork for PRs + +You maintain a personal fork on GitHub and use it as your push target +for PRs landing in upstream. `upstream` stays pull-only. + +```nix +lnbits-sensei.git.remotes = { + upstream = "https://github.com/lnbits/lnbits"; + fork = "git@github.com:/lnbits.git"; + extras = [ ]; +}; +``` + +What you get: + +- `upstream` for fetching releases / rebasing onto main. +- `fork` as the default push target. +- The upstream-PR helper (later pass) knows to push branches to `fork` + and open the compare URL against `upstream`. + +## 3. Multi-remote with a private host + +You also push to a private forgejo / gitea / codeberg mirror — for +internal review, a deployment pipeline, or just an off-GitHub backup. +The upstream + public-fork flow stays intact; the extras layer on top. + +```nix +lnbits-sensei.git.remotes = { + upstream = "https://github.com/lnbits/lnbits"; + fork = "git@github.com:/lnbits.git"; + extras = [ + { name = "internal"; url = "git@:/lnbits.git"; } + { name = "mirror"; url = "git@codeberg.org:/lnbits.git"; } + ]; +}; +``` + +What you get: + +- Same as pattern 2, plus the named extras as additional remotes. +- Each extra becomes a `git remote add ` on bootstrap; + pushing to them is opt-in (never the default push target). + +## Notes + +- `extras` is a list of `{ name; url; }` submodules — order is + preserved, and future fields (`pushUrl`, `mirror`) can be added + without breaking existing configs. +- The module is schema-only. The dev-env bootstrap script (later pass) + is what actually runs `git remote add` against a real checkout. It + is idempotent: re-running after editing `extras` reconciles the + on-disk remotes with the declared set. +- `fork = null` is the right value when you have no GitHub fork — + don't point it at a placeholder URL, the bootstrap script keys on + null to skip the `fork` remote entirely. diff --git a/modules/dev-env/docs/secrets-management.md b/modules/dev-env/docs/secrets-management.md new file mode 100644 index 0000000..08327c7 --- /dev/null +++ b/modules/dev-env/docs/secrets-management.md @@ -0,0 +1,110 @@ +# Secrets Management + +Layered approach to secrets in dev-env. None of this is enforced by the +module — these are conventions that pair well with the dev-env layout. + +| Context | Tool | Storage | +|---|---|---| +| Local dev | `pass` | `~/.password-store` (GPG encrypted) | +| NixOS servers | `sops-nix` + `age` | Encrypted in git | +| Repo secrets | `git-crypt` or `.sops.yaml` | Encrypted files in repo | + +The shared pre-commit hook (installed by `dev-env.gitHooks.enable`) +refuses to commit common secret patterns and unencrypted sops files. +See `dev-env/scripts/git-hooks/pre-commit` for the patterns. + +## Local dev with `pass` + +```bash +# pass is in dev-env's package set; gpg is in omni's defaults +gpg --gen-key # if you don't have one +pass init "your-email@example.com" + +pass insert dev/lnbits/admin-key +pass insert dev/postgres/password +pass insert dev/bitcoin/rpc-password + +pass dev/lnbits/admin-key # print +pass -c dev/lnbits/admin-key # to clipboard (clears after 45s) +export LNBITS_ADMIN_KEY=$(pass dev/lnbits/admin-key) # in scripts +``` + +## Server secrets with sops-nix + +### Initial setup + +```bash +age-keygen -o ~/.config/sops/age/keys.txt # generate your key +age-keygen -y ~/.config/sops/age/keys.txt # show your public key +# age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +``` + +### Per-project + +In each deploy host directory: + +```bash +cd ~/dev/deploy/unified/hosts/host5 + +cat > .sops.yaml << EOF +keys: + - &admin age1xxx... # you + - &host5 age1yyy... # the server's key + +creation_rules: + - path_regex: secrets/host5\.yaml$ + key_groups: + - age: + - *admin + - *host5 +EOF + +mkdir -p secrets +nvim secrets/host5.yaml # write the unencrypted file +sops -e -i secrets/host5.yaml # encrypt in place +``` + +The pre-commit hook will refuse to commit `secrets/host5.yaml` if it's +not encrypted. (False positive? `git commit --no-verify`.) + +### Using in NixOS + +```nix +{ config, pkgs, ... }: + +{ + imports = [ inputs.sops-nix.nixosModules.sops ]; + + sops.defaultSopsFile = ./secrets/host5.yaml; + sops.age.keyFile = "/var/lib/sops-nix/key.txt"; + + sops.secrets."lnbits/admin_key" = {}; + sops.secrets."postgres/password" = {}; + + services.lnbits = { + adminKeyFile = config.sops.secrets."lnbits/admin_key".path; + }; +} +``` + +### Bootstrapping a server's age key + +On each NixOS host (one-time): + +```bash +sudo mkdir -p /var/lib/sops-nix +sudo age-keygen -o /var/lib/sops-nix/key.txt +sudo chmod 600 /var/lib/sops-nix/key.txt +sudo age-keygen -y /var/lib/sops-nix/key.txt # add to .sops.yaml +``` + +## Best practices + +1. **Never commit unencrypted secrets.** The pre-commit hook helps but + isn't a substitute for paying attention. +2. **Rotate after team changes** — especially when removing keys. +3. **Different secrets per environment.** staging ≠ production. +4. **Backup your master keys.** GPG and age private keys are the only + thing standing between you and a total loss. +5. **No secrets in `.devenv.conf` / `/etc/dev-env/config.sh`.** Those + files are world-readable. diff --git a/modules/dev-env/docs/stack-overview.md b/modules/dev-env/docs/stack-overview.md new file mode 100644 index 0000000..e90f2eb --- /dev/null +++ b/modules/dev-env/docs/stack-overview.md @@ -0,0 +1,286 @@ +# Aiolabs stack — high-level overview + +Orientation map for new contributors. The substrate is **Omnixient** (this +repo): a NixOS desktop that declaratively materialises every project +worktree below via `modules/dev-env/` + the +[`aiolabs` preset](../presets/aiolabs.nix). + +```mermaid +flowchart TB + subgraph DEV["Omnixient — NixOS dev environment (this repo)"] + DEVENV["modules/dev-env + aiolabs preset
worktrees · tmux · git hooks
Forgejo origin + GitHub upstream forks"] + end + + subgraph DEPLOY["Deploy"] + DEPLOYFLAKE["server-deploy flake
host1 · host5 · host7 · host8
host4 · host3 · host6 · 484"] + end + + subgraph FRONT["User-facing frontends"] + direction LR + BITSPIRE["bitSpire
Bitcoin ATM frontend
KYC-free · Nostr-native"] + WEBAPP["webapp / 'AIO'
Vue 3 + TS · Electron + PWA
wallet · events · Nostr"] + end + + subgraph CORE["LNbits — Lightning backend (Python / FastAPI)"] + LNBITS["lnbits core
wallets · per-wallet API keys · extension runtime
nostr-native transport (kind-21000, NIP-44 v2)"] + end + + subgraph EXT["LNbits extensions — shared/extensions/"] + direction LR + SPIRE["spirekeeper
bitSpire operator dashboard"] + NREL["nostrrelay"] + NCLI["nostrclient"] + EVENTS["events"] + NMKT["nostrmarket"] + LIBRA["libra"] + LNURLP["lnurlp"] + REST["restaurant"] + end + + RELAYS(("Nostr relays
event bus")) + BTC["Bitcoin / Lightning
LND · CLN · Phoenixd · …"] + FAVA["Fava + Beancount
double-entry ledger"] + REF["Lightning.Pub (shocknet)
reference implementation for the
nostr-native Lightning interface"] + + DEV -. develops .-> CORE + DEV -. develops .-> FRONT + DEV -. drives .-> DEPLOY + DEPLOY -- ships --> CORE + DEPLOY -- ships --> FRONT + LNBITS -- hosts --> EXT + + %% On Nostr there are no point-to-point links: the nostr-native + %% parts all meet on the relay bus, addressed by pubkey. bitSpire↔ + %% LNbits (payments) and bitSpire↔spirekeeper (fleet) both resolve + %% through the relay, not over a direct edge. + BITSPIRE <-- "Lightning RPC + fleet (kind-21000)" --> RELAYS + SPIRE <-- "bitSpire fleet control" --> RELAYS + NCLI -- "LNbits ↔ relay" --> RELAYS + NREL -- "in-stack relay" --> RELAYS + EVENTS -- publishes --> RELAYS + NMKT -- marketplace --> RELAYS + WEBAPP -- subscribes --> RELAYS + + %% non-Nostr edges + WEBAPP -- REST + WebSocket --> LNBITS + LNBITS -- funding source --> BTC + LIBRA -- HTTP / JSON API --> FAVA + REF -. reference impl .-> LNBITS + + classDef reference stroke-dasharray: 4 4,opacity:0.65; + class REF reference; +``` + +## How to read this + +- **Omnixient** is where you *work*. Cloning it and running the dev-env + bootstrap materialises every box below as a worktree under + `~/dev/`, set up with the right Forgejo origin and GitHub upstream + fork. +- **LNbits** is the backend platform. Everything user-visible either + *is* an LNbits extension or *talks to* LNbits. It holds wallets, + funds them from a Bitcoin/Lightning source, runs the extension + runtime, and exposes both a classic HTTP/WebSocket API and a + **nostr-native transport** — signed-event RPC over a relay + (kind-21000, NIP-44 v2) with no bearer tokens on the wire. +- **Extensions** are Python modules dropped into LNbits' extension + runtime. They're cloned independently into `shared/extensions/` so + multiple LNbits worktrees (dev/main) can share them. +- **libra** is the unusual one: it doesn't reimplement double-entry + accounting, it speaks to **Fava**'s JSON API. Fava is the web UI on + top of **Beancount** plain-text ledgers. So libra = LNbits plugin + that delegates accounting to a separate Fava service. +- **webapp** ("AIO") is the polished, abstracted user frontend. It + treats LNbits as a backend; configured via `VITE_LNBITS_BASE_URL`. +- **bitSpire** is the Bitcoin ATM frontend — a KYC-free, Nostr-native + ATM. It talks to its Lightning backend over the **nostr-native + transport** on a relay rather than HTTP, so the kiosk holds no admin + tokens: the ATM's own Nostr key *is* its credential, and LNbits + auto-provisions its wallet on first contact from that signature. +- **spirekeeper** is the operator control dashboard for bitSpire + fleets, built as an LNbits extension. It owns the operator side of a + deployment: fleet and cassette-inventory configuration, commission / + DCA distribution (parsed from each cash-out's `Payment.extra`), + operator branding, and telemetry — all carried over Nostr. + spirekeeper and bitSpire never connect directly: like every + nostr-native pair in the stack, they meet on the relay, each + addressed by its own Nostr pubkey. +- **server-deploy** is a separate flake that pins each project's + source revision and builds NixOS configurations for the production + hosts (`host8`, `host7`, `host5`, …). + +> **Credit — Lightning.Pub.** LNbits' ATM-facing side — its +> nostr-native Lightning node-management interface — is modeled on +> [**Lightning.Pub**](https://github.com/shocknet/Lightning.Pub) +> (shocknet), the reference implementation for this pattern. bitSpire +> talks to LNbits the way it would talk to a Lightning.Pub node; the +> functionality lives in LNbits, but the shape of the interface is +> Lightning.Pub's. + +## Future direction: Nostr-native transport + +The `bitSpire ↔ LNbits` edge already runs on the nostr-native +transport — signed-event RPC over a relay, with Nostr's +key/signature model as the security primitive (auth, capabilities, +audit trail) instead of bearer tokens and TLS-only trust. The +longer-term direction is to move the *rest* of the stack's +inter-service edges the same way, so `webapp ↔ LNbits` and +`libra ↔ Fava` migrate from REST/WebSocket onto signed Nostr events +too. The proposed security structure is described in +[aiolabs/lnbits#9](https://git.atitlan.io/aiolabs/lnbits/issues/9). + +The nostrclient/nostrrelay extensions already make LNbits a +first-class Nostr participant; the issue tracks the broader +auth/permissions design that has to land before the rest of the stack +can follow. + +The same model is the candidate for **unified authentication** +across the whole platform: instead of each service running its own +account system (LNbits accounts, Authentik SSO, Forgejo logins, …), +a user's Nostr keypair becomes the identity and per-service +capabilities are granted as signed permission events. One identity, +one signing key, role-based access fanned out across LNbits, +Authentik, Forgejo and every other internal service. + +## Deployment model: location-portable instances + +The stack is designed to be **deployed anywhere** — every venue (a +co-living house, a makerspace, a community space, a bar running an +ATM) runs its own full instance: LNbits + the relevant extensions + +a local Nostr relay + supporting services. Each location is its own +sovereign tenant; nothing has to phone home to a central server. + +Because identity lives in the user's Nostr keypair (see above) +rather than in any per-location account, the user experience is +*roaming*: when someone walks into a new location they point their +client at that location's relay and the same app surface comes up, +populated with that location's data — their local membership, +balances, ledger position, events. Identity travels with the user; +data stays scoped to the place. + +The `server-deploy` flake already supports this shape — `host8`, +`host7`, `host4`, `host5` etc. are independent instances built +from the same source tree. + +## Target architecture (Nostr-native, location-portable) + +If everything above lands, the picture collapses dramatically. There +are no per-edge transports to draw and no per-service auth boxes — +just **one identity**, **one client surface**, and **N self-similar +location instances** that the client roams between by switching +relays. + +```mermaid +flowchart TB + subgraph USER["The user"] + KEY["🔑 Nostr keypair
identity · auth · permissions"] + CLIENT["Client surface
webapp · bitSpire · future apps"] + KEY -. signs every event .-> CLIENT + end + + subgraph LOC_A["Location A — e.g. Castle"] + RELAY_A(("Nostr relay")) + SVCS_A["LNbits + extensions
Fava / Beancount
Authentik · Forgejo
Bitcoin / Lightning"] + SVCS_A <-- Nostr events --> RELAY_A + end + + subgraph LOC_B["Location B"] + RELAY_B(("Nostr relay")) + SVCS_B["same stack
per-location data"] + SVCS_B <-- Nostr events --> RELAY_B + end + + subgraph LOC_N["Location N …"] + RELAY_N(("Nostr relay")) + SVCS_N["same stack
per-location data"] + SVCS_N <-- Nostr events --> RELAY_N + end + + CLIENT == currently connected ==> RELAY_A + CLIENT -. roam .-> RELAY_B + CLIENT -. roam .-> RELAY_N +``` + +Read this against the first diagram and the simplification is the +point: the tangle of REST/WebSocket arrows becomes a single +event-bus edge per service, the auth boxes disappear into the +keypair, and every location is the same shape. New locations are +just another copy of the bubble; new client apps are just another +signer using the same key. + +## The ATM as cash ↔ sats bridge + +The picture above is the *sovereign* half of the story. The other +half is that people still live in a **cash** economy — and at each +location the ATM (`bitSpire`) is the protected portal between the two +worlds. It sits inside the location instance and acts as a one-stop, +identity-preserving bridge: + +- **Cash stays anonymous.** Walking up to the ATM with a banknote + needs no account, no KYC, no Nostr key. Cash keeps its + permissionless, locally-spendable nature — useful at every local + merchant whether or not they speak Lightning. +- **Onramp: cash → sats.** The ATM credits the user's LNbits wallet + at this location. The wallet is **custodial for now** — the + operator holds the sats, the way a bartender holds a cash tab. The + user trusts the operator to "carry the cash" between visits; the + trust is bounded by the escape hatch below. +- **Offramp: sats → cash.** The user can spend that location-wallet + balance back out as cash through the same ATM, or use it directly + with local merchants and LNbits extensions at the venue. No KYC + is required in either direction. +- **Escape to self-custody.** The custodial wallet is a checkpoint, + not a destination. One tap pulls sats out to the user's **cold + on-chain Bitcoin wallet** or to their **own self-custodied + Lightning wallet** — so anyone uncomfortable trusting the operator + can leave at any time with their full balance. + +```mermaid +flowchart LR + subgraph FIAT["Local fiat economy"] + CASH["💵 Cash
anonymous · permissionless
spendable at local merchants"] + end + + subgraph LOC["Location instance"] + direction TB + ATM["🏧 bitSpire ATM
protected portal
no KYC"] + WALLET["⚡ LNbits wallet
custodial · operator-held
Nostr key = login"] + STACK["LNbits + extensions
local Nostr relay"] + ATM <-- "nostr-native transport" --> WALLET + WALLET --- STACK + end + + subgraph SELF["User self-custody (off-platform)"] + COLD["❄️ cold on-chain
Bitcoin wallet"] + OWNLN["⚡ own Lightning
wallet / node"] + end + + CASH <-- "cash ↔ sats" --> ATM + WALLET == "withdraw on-chain" ==> COLD + WALLET == "withdraw via LN" ==> OWNLN +``` + +The ATM is therefore the **gateway** that lets a user move value +freely between each location's legacy cash economy and the +Nostr-native ecosystem — without surrendering the anonymity that +cash and a Nostr keypair each provide on their own side of the +boundary. Cash works as cash; sats work as sats; the ATM is the +seam, and crossing it doesn't cost you your identity. + +## Where the source lives + +| Box | Path under `~/dev/` | Origin (Forgejo) | +|----------------------|--------------------------------|------------------------------------| +| Omnixient | (this repo, `/etc/nixos`) | `aiolabs/omnixient` | +| LNbits | `lnbits/{dev,main}` | `aiolabs/lnbits` (no fork) | +| Extensions | `shared/extensions/` | `aiolabs/` | +| spirekeeper | `shared/extensions/spirekeeper`| `aiolabs/spirekeeper` | +| webapp | `webapp/` | `aiolabs/webapp` | +| bitSpire | `bitspire/bitspire/{main,dev}` | `aiolabs/bitspire` | +| atm-tui | `bitspire/atm-tui` | `aiolabs/atm-tui` | +| server-deploy | `deploy/server-deploy` | `aiolabs/server-deploy` | +| nips (reference) | `nostr-protocol/nips` | nostr-protocol/nips (read-only) | +| regtest | `local/docker/regtest` | `aiolabs/regtest` | + +> Lightning.Pub is kept as a read-only reference mirror under +> `~/dev/refs/` (see `refs.toml`), not as a runtime component. diff --git a/modules/dev-env/docs/upstream-prs.md b/modules/dev-env/docs/upstream-prs.md new file mode 100644 index 0000000..056accc --- /dev/null +++ b/modules/dev-env/docs/upstream-prs.md @@ -0,0 +1,181 @@ +# Contributing to Upstream Projects + +How to contribute fixes and features back to upstream projects (lnbits, +lamassu, nix-bitcoin, Lightning.Pub) using the worktree-based workflow +provided by `dev-env`. + +## Prerequisites (one-time, per repo) + +1. **Fork the upstream repo on GitHub.** Click "Fork" on the upstream's + GitHub page (e.g. github.com/lnbits/lnbits) to create your personal + copy. + +2. **Set your GitHub username** in the dev-env config (or override + per-project): + + ```nix + dev-env.github.forkUser = "your-github-username"; + ``` + +3. **Re-run `dev-env-bootstrap`.** It adds the `github-fork` remote to + the bare repo derived from `git@github.com:/.git`. + Verify with: + + ```bash + git -C ~/dev/repos/lnbits.git remote -v + # origin forgejo@git.atitlan.io:aiolabs/lnbits.git (fetch) + # upstream https://github.com/lnbits/lnbits (fetch) + # github-fork git@github.com:your-github-username/lnbits.git (fetch) + ``` + +## Quick reference + +| Command | Alias | Description | +|---|---|---| +| `git-pr-branch ` | `prb` | Create PR worktree from upstream/main | +| `git-pr-cleanup ` | `prc` | Remove worktree after PR merges | +| `git-pr-list` | `prl` | List active PR worktrees | +| `prs` | — | `cd ~/dev/upstream-prs` | + +## Workflow + +### 1. Create the PR branch + +```bash +prb lnbits fix-invoice-validation +# Fetches upstream +# Creates 'fix-invoice-validation' from upstream/main +# Adds worktree at ~/dev/upstream-prs/lnbits-fix-invoice-validation +``` + +### 2. Make changes + +```bash +cd ~/dev/upstream-prs/lnbits-fix-invoice-validation +git status # On branch fix-invoice-validation +nvim src/some_file.py +pytest tests/ +git commit -am "Fix invoice validation for zero-amount invoices" +``` + +### 3. Push to your GitHub fork + +```bash +git push github-fork fix-invoice-validation +``` + +### 4. Open the PR on GitHub + +Go to upstream (e.g. github.com/lnbits/lnbits). You'll see the +"fix-invoice-validation had recent pushes" banner. Click "Compare & +pull request" and fill in title/description. + +### 5. Address review feedback + +```bash +cd ~/dev/upstream-prs/lnbits-fix-invoice-validation +nvim src/some_file.py +git commit -am "Address review: add input sanitization" +git push github-fork fix-invoice-validation +``` + +### 6. Cleanup after merge + +```bash +prc lnbits fix-invoice-validation +# Removes worktree, deletes local branch +``` + +## Layout + +``` +~/dev/ +├── repos/ +│ └── lnbits.git # bare repo, three remotes: +│ ├── origin # forgejo (your team's fork) +│ ├── upstream # github (lnbits/lnbits) +│ └── github-fork # github (your-github-username/lnbits) +│ +├── lnbits/ # team-fork worktrees +│ ├── dev/ +│ └── main/ +│ +└── upstream-prs/ + └── lnbits-fix-invoice-validation/ # transient PR worktree +``` + +## Git remotes explained + +| Remote | Points to | Used for | +|---|---|---| +| `origin` | Forgejo | Your team's fork — `main`, `dev`, `feature/*` | +| `upstream` | GitHub (original) | Read-only; fetch latest upstream changes | +| `github-fork` | GitHub (your fork) | Write-only target for PR branches | + +## Common scenarios + +### Sync with upstream before starting + +`prb` does this for you, but if you need to manually: + +```bash +cd ~/dev/repos/lnbits.git +git fetch upstream +``` + +### Rebase on latest upstream mid-PR + +```bash +cd ~/dev/upstream-prs/lnbits-fix-invoice-validation +git fetch upstream +git rebase upstream/main +git push github-fork fix-invoice-validation --force-with-lease +``` + +### Multiple PRs for the same repo + +```bash +prb lnbits fix-invoice-validation +prb lnbits add-webhook-support +prb lnbits update-deps +prl # list all +``` + +### Abandon a PR + +```bash +prc lnbits fix-invoice-validation # same as cleanup +``` + +## Best practices + +1. **One PR per feature/fix.** Keep PRs focused and reviewable. +2. **Branch from upstream/main.** `prb` enforces this. +3. **Clear commit messages.** What and why, not just what. +4. **Test before pushing.** +5. **Cleanup after merge.** Don't accumulate stale worktrees. +6. **Rebase, not merge.** Keeps a clean history. + +## Troubleshooting + +**"github-fork remote not found"** + +```bash +git -C ~/dev/repos/lnbits.git remote add github-fork \ + git@github.com:$GITHUB_FORK_USER/lnbits.git +``` + +Or re-run `dev-env-bootstrap` after setting `dev-env.github.forkUser`. + +**"Permission denied" pushing to github-fork** + +- Make sure you've forked the repo on GitHub. +- Check `ssh -T git@github.com` succeeds. +- Verify the URL: `git -C ~/dev/repos/lnbits.git remote -v`. + +**"Branch already exists"** + +```bash +prc lnbits fix-x # cleanup first +prb lnbits fix-x # then recreate +```