aiolabs/omnixient
9
0
Fork 0
Code Issues 5 Pull requests Projects Releases Packages Wiki Activity Actions

docs(dev-env): aiolabs stack overview and lnbits workflow notes

Browse source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Padreug 2026-06-28 06:48:37 +02:00
commit 346042f1c5
8 changed files with 1276 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

143
modules/dev-env/docs/lnbits-extension-dev.md Normal file
View file

@ -0,0 +1,143 @@
# lnbits extension development
Reference for building and maintaining LNbits extensions — the parts
that catch first-time extension authors and the patterns worth
adopting once you maintain a fork-modified extension.
## Auth decorators
Easy to confuse. Different scopes:
| Decorator | Auth scope | Returns | Use for |
|---|---|---|---|
| `require_invoice_key` | Wallet invoice key — read access | `Wallet` | Read endpoints (balance, payment history) callable with a user's invoice key |
| `require_admin_key` | Wallet admin key — write to **own wallet only** | `Wallet` | Endpoints that create payments / modify wallet state |
| `check_admin` | **LNbits instance admin** (super_user + `lnbits_admin_users`), Bearer token | `Account` | Cross-user / admin-only operations |
| `check_super_user` | LNbits super user only, Bearer token | `Account` | Operations restricted to the single super_user |
**`require_admin_key` ≠ `check_admin`.** This is the most common
misuse. `require_admin_key` is a *wallet*-level write key — every
user has one for each of their own wallets. `check_admin` is
*instance* admin access. Endpoints that operate on other users' data
or change global settings need `check_admin`, not `require_admin_key`.
## Testing
Use **FakeWallet**
(`LNBITS_BACKEND_WALLET_CLASS=FakeWallet`) for testing extension CRUD,
API endpoints, and UI changes. Spin up the full regtest stack only
when end-to-end Lightning payment behavior is actually under test.
**Why:** regtest takes time to start (docker build + multiple LND/CLN
containers), uses real resources, and adds no fidelity for non-payment
flows. FakeWallet makes payments succeed instantly with no network —
ideal for testing the surrounding logic.
If you ship a `dev` CLI (see lnbits-sensei's pattern), keep
`--fakewallet` as the default mode for this reason.
## Fork-migrations pattern (`migrations_fork.py`)
Long-lived forks need to add schema columns and tables on top of
upstream's. The naive approach — append fork-only migration functions
to `migrations.py` — guarantees merge conflicts on every upstream
rebase. The pattern below sidesteps that by keeping `migrations.py`
byte-identical to upstream and putting fork deltas in a sibling file.
> **Note:** this pattern requires a patch to LNbits core's
> `migrate_extension_database()` that loads `migrations_fork.py` under
> a `<ext.id>_fork` key in the `dbversions` table. As of 2026 that
> patch is fork-internal (an upstream PR is the natural follow-up).
> If you maintain a fork that ships this patch, the rest of this
> section is the user-facing pattern.
### Architecture facts
- `dbversions` lives in the **core LNbits DB** (`database.sqlite3`),
not per-extension. Schema: `(db TEXT PRIMARY KEY, version INT)`.
`update_migration_version` is an INSERT-OR-UPDATE, so a new
`<ext>_fork` row appears on first run with no schema migration
needed core-side.
- Extension data tables live in `ext_<id>.sqlite3` (SQLite) or a
Postgres schema named after `<id>`. Created lazily on first
`Database.connect()` via `ATTACH` (SQLite) or `CREATE SCHEMA`
(Postgres).
- The version-bump connection is routed based on `db.schema`: `None`
means a core migration (same connection), set means an extension
(opens a fresh `core_db.connect()` to write `dbversions`). See
`lnbits/core/helpers.py:run_migration`.
- **No cross-DB atomicity.** Extension migration writes commit to
`ext_<id>.sqlite3`; the `dbversions` upsert commits to
`database.sqlite3`. If the extension write succeeds and the
dbversions write fails, the migration is orphaned — re-runs on
next startup. **Every fork migration MUST be idempotent** (use
an `_alter_add_column_safe` wrapper that swallows
duplicate-column errors, `CREATE TABLE IF NOT EXISTS`, etc.).
Self-healing covers the orphan case.
### Squash recipe (adopting the pattern on an existing fork)
If your fork has accumulated fork-only migrations interleaved into
`migrations.py`:
1. **Restore `migrations.py` to upstream-byte-identical content.**
Drop all fork-only `m{NNN}_*` functions and any helper added only
for them.
2. **Create `migrations_fork.py`** with a SINGLE
`m001_<your-tag>_<ext>_schema` function that idempotently applies
every fork-only schema delta the old migrations used to do. One
readable file forever.
3. The squash uses `_alter_add_column_safe` per ALTER and
`CREATE TABLE IF NOT EXISTS` per table — no-ops cleanly on
installs that already ran the old fork migrations.
### One-time fix on existing installs adopting the pattern
Installs that previously ran the old fork migrations have their
`dbversions['<ext>']` row ahead of upstream (e.g. `events|11` after
fork-only `m007`–`m011`). After moving to `migrations_fork`, the
next upstream rebase that adds e.g. `m007_add_allow_fiat` would
compare `7 > 11 → false` and **silently skip** the new upstream
migration.
**Reset the row to match upstream's actual migration count** before
the rebase lands:
```sql
-- Run against the CORE DB (database.sqlite3), not the extension DB.
UPDATE dbversions SET version = <upstream-max> WHERE db = '<ext>';
```
For containerized dev where the file is root-owned inside the
container:
```bash
docker compose exec lnbits python3 -c "import sqlite3; \
c = sqlite3.connect('database.sqlite3'); \
c.execute(\"UPDATE dbversions SET version = <N> WHERE db = '<ext>'\"); \
c.commit()"
```
### Upstream-overlap scenario at rebase time
If upstream eventually adds a schema change you already carry in
`migrations_fork.py` (e.g. they add the same `ALTER TABLE … ADD COLUMN
bar` you shipped), the next rebase creates a problem: fresh installs
work (your `_alter_add_column_safe` guards swallow the dup), but
**existing installs crash** when upstream's now-redundant migration
runs without an idempotency guard (upstream rarely uses them).
Mitigation at rebase time:
1. **Prune the redundant block from `migrations_fork.py`** so future
fresh installs get the column from upstream's migration.
2. **Pre-deploy `dbversions` surgery on every affected install**:
`UPDATE dbversions SET version = <upstream-max> WHERE db = '<ext>'`
so the loader skips the now-overlapping upstream migration.
Do both — (1) keeps future fresh installs clean, (2) keeps existing
installs alive through the deploy.
**Don't** patch upstream's migration with idempotency guards in your
fork. That breaks the "migrations.py == upstream byte-identical"
property and reintroduces every-rebase conflicts.

105
modules/dev-env/docs/lnbits-frontend-gotchas.md Normal file
View file

@ -0,0 +1,105 @@
# lnbits frontend gotchas
LNbits ships its UI with **Vue 3 + Quasar 2 as UMD globals** — no
build step, plain Jinja templates with per-page JS. This applies
across lnbits core (`lnbits/templates/*.html`), every extension
(`<ext>/templates/`), and every fork that doesn't restructure the
frontend stack. The UMD load model has several traps that don't
manifest under the build-step model most Vue tutorials assume.
## No self-closing tags
Per [Quasar's UMD usage rules](https://quasar.dev/start/umd/#usage),
components must use the explicit-close form:
```html
<!-- correct -->
<q-input v-model="foo" label="Foo"></q-input>
<q-btn @click="bar" label="Bar"></q-btn>
<!-- wrong — silently broken in UMD/no-build mode -->
<q-input v-model="foo" label="Foo" />
<q-btn @click="bar" label="Bar" />
```
**Why:** UMD-loaded templates are parsed by the browser's HTML parser,
not Vue's compiler. The HTML parser doesn't honor self-close on
non-void elements (per the HTML spec). The close tag gets implied at
the wrong place, nesting breaks silently, and subsequent siblings end
up nested inside the prior component.
Self-closing is fine in `.vue` SFCs (the build step rewrites them
before the browser sees anything), so if you copy a snippet from a
Vue SFC repo into an LNbits template, **expand all self-closing tags
before saving**.
## CSS specificity trap
LNbits applies its own theme overrides on Quasar's typography
utilities (`.text-caption`, `.text-grey-*`, etc.) with `!important`.
Class-based CSS rules in an extension page — *even with `!important`* —
lose this fight unless your selector is strictly more specific than
the upstream rule.
**Rule:** for per-element typography/color overrides on LNbits pages,
reach for Vue `:style` bindings (or static `style="..."` attrs), not
`<style>` blocks targeting Quasar utility classes:
```html
<!-- ✗ likely loses to upstream's !important rule -->
<style>
.text-caption.my-fix { color: #ff0000 !important; }
</style>
<!-- ✓ inline style wins without an arms race -->
<span :style="{ color: '#ff0000' }">…</span>
<span style="color: #ff0000">…</span>
```
Background/border tweaks at card-level via class are fine — the trap
is specifically the typography utilities (`text-*`) and Quasar's
color utilities.
## Cache busting
Static assets are served with `?v={server_startup_time}` appended
(see `static_url_for` in `lnbits/helpers.py`). Consequences:
- **Bumping JS requires a server restart.** Reloading the browser
doesn't help if `?v=` hasn't changed — the browser keeps serving
the cached file.
- **Jinja templates re-render on every request** (the `?v=` is only
on static assets). No restart needed for template edits — just
refresh.
If a browser keeps serving stale JS after a restart, hard-refresh
(`Ctrl+Shift+R`) to bypass the HTTP cache.
## Dark-mode color discipline
LNbits's dark theme inverts text colors on most surfaces but **not**
on `bg-{color}-1` pale-background utilities. Result: a `bg-red-1`
without an explicit text color renders white-on-cream under dark
theme — basically invisible.
**Rule:** pair every pale-background utility with an explicit dark
text class:
```html
<!-- ✗ unreadable on dark theme -->
<div class="bg-red-1 q-pa-md">Warning</div>
<!-- ✓ explicit text class survives theme switch -->
<div class="bg-red-1 text-grey-9 q-pa-md">Warning</div>
```
Same for `bg-green-1`, `bg-blue-1`, `bg-amber-1`, etc. The `text-grey-9`
choice is the safe default; pick a darker shade if you want stronger
contrast.
## When in doubt
Test under both light and dark themes (Quasar's theme toggle is at
the top of every LNbits page once you're logged in). Most of the
above gotchas are silent under one theme and obvious under the other
— don't ship UI without flipping the toggle at least once.

128
modules/dev-env/docs/lnbits-upstream-flow.md Normal file
View file

@ -0,0 +1,128 @@
# `lnbits/lnbits` — Upstream Development Flow
Reference for [`github.com/lnbits/lnbits`](https://github.com/lnbits/lnbits).
Verified against git history on 2026-05-18; bump the date when you
re-verify, and patch this doc if the model has drifted.
## Branch model
Two long-lived branches:
| Branch | Role | How it moves |
|---|---|---|
| `dev` | Integration / staging | One squash-merge commit per PR. Linear. |
| `main` | Release | A non-fast-forward merge of `dev` at each release. |
`main` never receives feature PRs directly. **Every change reaches
`main` through `dev`.**
## PR flow → `dev`
1. Contributor opens a PR targeting `dev` (not `main`).
2. Maintainer squash-merges via the GitHub UI.
3. The resulting commit on `dev` has:
- exactly one parent (linear history),
- subject ending in `(#NNNN)` — the squash-merge signature,
- lowercase conventional-commit prefix: `feat:`, `fix:`, `chore:`,
`chore(deps):`, `docs:`, `ci:`, `test:`, `refactor:`.
4. CI runs on the PR; nothing else is required between merge and the
commit appearing on `dev`.
Example chain on `dev`:
```
c9c68bd8 Fix: Use default reaction on bootstrap (#3965)
810a1372 fix: tighten agents file (#3966)
36d696b2 Fix: wrong use of `in` operator (#3960)
8b426efa test: add pyinstrument profiler (#3955)
```
Each is a single squashed commit. No merge commits inside `dev`.
## Release flow → `main`
When `dev` is ready to ship:
1. A release-candidate version bump lands on `dev` as a normal PR:
`chore: update to version vX.Y.Z-rcN (#NNNN)`.
2. Validation happens against the RC.
3. A final version-bump PR lands on `dev`:
`chore: update to version vX.Y.Z (#NNNN)`.
4. A maintainer runs the release merge locally:
```bash
git checkout main
git pull --ff-only origin main
git merge dev # true (non-FF) merge, default message
git push origin main
git tag vX.Y.Z # tag the merge commit or the bump commit
git push origin vX.Y.Z
```
5. The resulting merge commit on `main` has:
- **two parents** (prior `main` tip + `dev` tip),
- subject exactly `Merge branch 'dev'` (git default when on `main`),
- **not** authored via the GitHub PR-merge UI (that would produce
`Merge pull request #N from …`).
Because `main` typically carries a stray release-bump commit that isn't
on `dev`, the histories have diverged and git is forced into a true
merge. `--no-ff` is not needed for that reason.
## Reading the history
```bash
# Release log (one entry per release):
git log main --first-parent --oneline
# Full changelog leading into the next release:
git log dev --oneline
# Verify a merge is a true non-FF merge:
git log <sha> -1 --format='%P' # two parent hashes = true merge
```
## Diagram
```
(PRs squash-merged one at a time)
│
▼
dev: ── A ── B ── C ── D ── E ── F (rc1) ── G ── H (v1.5.4)
╲
╲ (true merge)
╲
main: ────────── prev release ─────────────── X ─────────── M
▲
│
Merge branch 'dev'
tag: v1.5.4
```
## Implications for contributors
- **Base PRs on `dev`.** PRs against `main` will not be accepted.
- **Use lowercase conventional-commit titles.** Verified stable across
the last 25+ merged PRs.
- **Don't expect `main` to move between releases.** It only advances
when a maintainer cuts a release merge.
- **Tagging is on `main`.** Consumers pinning to a tag get the
released state, never a `dev` snapshot.
## Adapting this in your own fork
If you maintain a long-lived fork of `lnbits/lnbits` for production use,
you'll likely want to:
1. Mirror upstream's `dev` / `main` split — easier to track upstream
merges back into your fork.
2. Adopt a version-suffix convention that surfaces fork identity in
tags and the packaged `pyproject.toml` version, e.g.
`v<upstream>-<your-tag>.<N>`. This makes it unambiguous in logs and
deployed-package metadata that you're running fork-modified code.
3. Decide whether you also need pre-release channels (`-rcN`, `-devN`)
on top of upstream's; useful if your fork ships to staging hosts
before promotion.
Specifics depend entirely on your team's needs — this scaffold
deliberately doesn't prescribe a fork-versioning scheme.

243
modules/dev-env/docs/lnbits-workspace-notes.md Normal file
View file

@ -0,0 +1,243 @@
# lnbits workspace notes
Practical reference for day-to-day work in an lnbits dev environment.
Collected gotchas, conventions, and design constraints that have
repeatedly surprised people. Not a tutorial — assumes you're already
running lnbits and contributing or extending it.
## Pick a non-default port for your local dev server
LNbits defaults to `:5000`. That collides with macOS AirPlay Receiver,
common docker-compose stacks, and other dev tools. Pick something
unambiguous (many forks settle on `:5001`) and stick to it across:
- `settings.nix` → `lnbits.port`
- any MCP server config you wire against your dev instance
- bookmarks, env files, CI configs
Once you wire it in three places, switching the port retroactively is
mostly find-and-replace pain. Decide early.
## LNBITS_SRC and docker-compose build context
If you run a docker-compose dev stack (regtest or otherwise) that
builds lnbits from a local checkout, the `Dockerfile` typically reads
from `${LNBITS_SRC:-/some/default}`.
The trap: **commits to your day-to-day worktree don't reach the dev
image if `LNBITS_SRC` is currently pointed elsewhere** (e.g. at a
feature-branch worktree you were testing). Even
`docker compose build --no-cache` happily rebuilds from the wrong
checkout.
Sanity-check before assuming a rebuild picked up your change:
```sh
docker compose config | grep -A2 lnbits
# look for the resolved `context:` path
```
If it's pointing somewhere stale, either cherry-pick your commit onto
the active build branch or flip `LNBITS_SRC` and rebuild.
## Extension folder is the install target — "Upgrade" wipes forks
If your dev setup mounts an extension checkout directly into the lnbits
container (e.g. `~/dev/shared/extensions/` → `/shared`, with
`LNBITS_EXTENSIONS_PATH=/shared`), **the extension git checkout IS the
installed extension**. There's no separate copy.
Consequence: clicking **"Upgrade"** in the LNbits UI on an extension
that's mounted from a fork checkout will:
1. Download the catalog tarball.
2. Extract it directly over the mounted directory.
3. **Wipe `.git`**, replace every file with the catalog version, and
silently discard any local changes / fork patches.
If this happens, recover with `git clone <your-fork-url> <dir>` over
the wiped directory. Mitigations:
- Set an extension version in your fork that always sorts above the
catalog's so "Upgrade" never thinks the catalog is newer.
- Or don't expose the catalog upgrade UI to a workflow that has
mounted-fork extensions.
## Settings precedence: `.env` seeds the DB on first boot, then the DB wins
This one trips up most people who deploy lnbits declaratively (via NixOS,
docker-compose, ansible, …). Verified 2026-05-24 against upstream
`lnbits/main`.
LNbits has two sources of truth for settings depending on lifecycle.
**On boot, when `lnbits_admin_ui=True`** (the default):
1. Read DB row via `get_super_settings()`
(`lnbits/core/services/users.py:236`).
2. If the DB row is empty → seed it from `.env` via
`init_admin_settings()`. First-boot only.
3. `update_cached_settings(settings_db.dict())` overwrites the
in-memory `Settings` with the DB row. **`.env` values loaded by
Pydantic at startup are clobbered.**
**Practical consequence:** once an instance has booted once, editing
`.env` and restarting **changes nothing** for editable fields. They can
only be changed via:
- The Admin UI (`PUT /api/v1/settings`, gated by `check_admin`), or
- Clearing the relevant rows in the `system_settings` table in the
core DB.
**Exceptions where `.env` still wins on every boot:**
- `super_user` — env overrides DB explicitly in `users.py`.
- `lnbits_admin_ui=False` — the whole DB-load block is skipped; env
stays authoritative (there's no Admin UI to populate the DB anyway).
- All `ReadOnlySettings` fields (defined in `lnbits/settings.py` —
`EnvSettings` / `PersistenceSettings` / `SuperUserSettings` /
`ExtensionsInstallSettings`). Concretely:
- `host`, `port`
- `lnbits_path`, `lnbits_data_folder`, `lnbits_extensions_path`
- `lnbits_database_url`
- `auth_secret_key`, `first_install_token`
- `lnbits_title` (the API title — NOT `lnbits_site_title`, which is
editable)
- `lnbits_admin_ui` itself
- `lnbits_allowed_funding_sources` (the *list of which sources can be
enabled* — the per-source credentials live in `FundingSourcesSettings`
→ `EditableSettings` and ARE DB-frozen)
`update_cached_settings` skips any key in `readonly_variables`, so
env-loaded values for these fields survive the DB-load overwrite.
**Editable** settings — site title/tagline, theme, watchdog thresholds,
fee defaults, rate limits, all per-funding-source credentials, the
whole Admin UI form — get DB-frozen after first boot.
### `LNBITS_FIRST_INSTALL_TOKEN` rotation ≠ "reset settings to env"
Rotating the first-install token creates a new super_user account with
a fresh random UUID and re-enables the `/first_install` endpoint. It is
an **escape hatch for a locked-out admin** to re-claim the instance
with a new username/password. It does NOT refresh any settings values
from `.env` — by the time it runs, the DB row has already overwritten
cached `settings`, and `init_admin_settings()` then upserts those same
DB values back, so no env values flow through.
### Deploy-side implication
If you manage lnbits config declaratively (NixOS module, ansible role,
docker-compose env file), **editable env vars only take effect on a
fresh install** (empty `settings` table). For an existing deployment,
changing them in your declarative source and redeploying won't change
runtime behavior — you have to edit through the Admin UI OR clear the
relevant rows in `system_settings`.
The "set it in nix, redeploy, done" mental model only works for
`ReadOnlySettings` fields. Anything an admin can edit in the UI is
DB-frozen post-first-boot. Plan your deploy story accordingly:
- For `ReadOnlySettings` (host, port, paths, secret_key, …):
declarative source is authoritative on every boot. Reproducible.
- For `EditableSettings` (site title, fees, funding-source creds, …):
declarative source is a *seed*, not authoritative. To re-seed an
existing instance, you must explicitly clear the DB row.
## Nostr key handling — don't persist plaintext nsecs
As of 2026, LNbits's upstream account model server-generates a Nostr
private key (`accounts.prvkey`) for every new account and stores it
**plaintext in the DB**. The key is returned to clients over HTTPS via
auth endpoints. A DB snapshot, backup leak, or curious operator is a
wholesale identity compromise for every user.
If your fork or extensions touch user Nostr keys, design around this.
A defensible shape:
- A `NostrSigner` abstraction with at least three implementations:
- `LocalSigner` — server-side key, but envelope-encrypted at rest.
User unlocks per session.
- `RemoteBunkerSigner` — NIP-46 connection to a bunker the user
runs themselves. Server never sees the private key.
- `ClientSideOnlySigner` — the server only knows the user's
public key. Signing happens entirely in the browser / NIP-07
extension / hardware signer.
- Optionally a `DelegatedSigner` via NIP-26 for accounts that want
a server-side signer for low-value operations while keeping
high-value ops client-side.
Also audit every server-side Nostr key your extensions persist
(merchant signing keys, notification keys, transport keys, …) and
either encrypt at rest or document why ephemeral is fine.
The general principle: **prefer "don't store the key" when possible.**
Let the user's signer (browser extension, hardware device, NIP-46
bunker) do the signing.
## CLINK is Lightning.Pub-specific, not LNbits
Shocknet's [CLINK protocol](https://github.com/shocknet/CLINK) — the
Nostr-native payment flow with event kinds 21001 (offer/noffer),
21002 (debit/ndebit request), 21003 (debit response) — lives inside
`Lightning.Pub`, not LNbits. LNbits as of 2026 has zero CLINK
knowledge.
If you're thinking about `ndebit` / `noffer` semantics in an lnbits
context:
- They're Lightning.Pub primitives. LNbits doesn't implement them.
- Building "equivalent outcomes" in LNbits typically goes through
LNURL + payment-hash subscriptions + opaque `extra` payloads
rather than CLINK-style events. Different primitives, similar
effective semantics.
- If you genuinely need CLINK in lnbits, it's a new transport
module, not a tweak to existing code paths. NIP-44 plumbing in
the nostr-transport surface is the natural building block.
Don't conflate the two — keep CLINK semantics on the Lightning.Pub
side of any cross-system design.
## Fork versioning — surface your fork in the version string
If you maintain a long-lived fork (especially one running in
production), the released version string should make it obvious that
fork-modified code is what's running. Consumers of the package, log
readers, and bug reporters all benefit.
A common pattern:
- **Git tag:** `v<upstream-version>-<your-tag>.<N>`
e.g. `v1.5.4-aio.1`, `v1.5.4-aio.2`, …
- **`pyproject.toml` `version`:** `<upstream-version>+<your-tag>.<N>`
e.g. `1.5.4+aio.1`.
The two spellings differ because PEP 440 doesn't accept
hyphen-with-arbitrary-suffix as a valid package version, but the local-
version form (`+<tag>.<N>`) is valid. Extensions don't have the PEP 440
constraint (their `config.json` is freeform), so the tag form works
everywhere except `pyproject.toml`.
Decide your own tag string and stick to it. The key property is that
`importlib.metadata.version("lnbits")` and the LNbits UI footer make
it visible to anyone looking.
For the upstream lnbits branch / release model these tags sit on top
of, see [lnbits-upstream-flow.md](lnbits-upstream-flow.md).
## Reading the codebase fast
For mass-grep and reference reading, point a tools-friendly mirror of
the lnbits source at a known location (see your `~/dev/refs/` setup if
you use one). Key paths once you have a checkout:
- `lnbits/core/services/` — core money flows. Start here for invoice
/ payment / wallet logic.
- `lnbits/wallets/` — backend implementations. One file per
`LNBITS_BACKEND_WALLET_CLASS` value.
- `lnbits/extensions/` — first-party extensions. Third-party
extensions follow the same shape (`config.json`, `views.py`,
`migrations.py`, etc.).
- `git log --first-parent dev --oneline` — see the squash-merge
sequence into `dev` (see lnbits-upstream-flow.md for why
`--first-parent` matters here).

80
modules/dev-env/docs/remotes.md Normal file
View file

@ -0,0 +1,80 @@
# Remote topology
`lnbits-sensei.git.remotes` (declared in `modules/git/remotes.nix`)
abstracts how the local LNbits checkout is wired to its git remotes.
Three patterns cover almost every workflow.
## 1. Upstream-only
You read upstream, never push. Good for a read-only dev box, a CI
runner, or initial exploration before you've decided to contribute.
```nix
lnbits-sensei.git.remotes = {
upstream = "https://github.com/lnbits/lnbits";
fork = null;
extras = [ ];
};
```
What you get:
- `upstream` remote pointing at canonical lnbits/lnbits.
- No `fork` remote — the bootstrap script skips it when `fork` is null.
- No extras.
## 2. GitHub fork for PRs
You maintain a personal fork on GitHub and use it as your push target
for PRs landing in upstream. `upstream` stays pull-only.
```nix
lnbits-sensei.git.remotes = {
upstream = "https://github.com/lnbits/lnbits";
fork = "git@github.com:<you>/lnbits.git";
extras = [ ];
};
```
What you get:
- `upstream` for fetching releases / rebasing onto main.
- `fork` as the default push target.
- The upstream-PR helper (later pass) knows to push branches to `fork`
and open the compare URL against `upstream`.
## 3. Multi-remote with a private host
You also push to a private forgejo / gitea / codeberg mirror — for
internal review, a deployment pipeline, or just an off-GitHub backup.
The upstream + public-fork flow stays intact; the extras layer on top.
```nix
lnbits-sensei.git.remotes = {
upstream = "https://github.com/lnbits/lnbits";
fork = "git@github.com:<you>/lnbits.git";
extras = [
{ name = "internal"; url = "git@<your-forgejo>:<org>/lnbits.git"; }
{ name = "mirror"; url = "git@codeberg.org:<you>/lnbits.git"; }
];
};
```
What you get:
- Same as pattern 2, plus the named extras as additional remotes.
- Each extra becomes a `git remote add <name> <url>` on bootstrap;
pushing to them is opt-in (never the default push target).
## Notes
- `extras` is a list of `{ name; url; }` submodules — order is
preserved, and future fields (`pushUrl`, `mirror`) can be added
without breaking existing configs.
- The module is schema-only. The dev-env bootstrap script (later pass)
is what actually runs `git remote add` against a real checkout. It
is idempotent: re-running after editing `extras` reconciles the
on-disk remotes with the declared set.
- `fork = null` is the right value when you have no GitHub fork —
don't point it at a placeholder URL, the bootstrap script keys on
null to skip the `fork` remote entirely.

110
modules/dev-env/docs/secrets-management.md Normal file
View file

@ -0,0 +1,110 @@
# Secrets Management
Layered approach to secrets in dev-env. None of this is enforced by the
module — these are conventions that pair well with the dev-env layout.
| Context | Tool | Storage |
|---|---|---|
| Local dev | `pass` | `~/.password-store` (GPG encrypted) |
| NixOS servers | `sops-nix` + `age` | Encrypted in git |
| Repo secrets | `git-crypt` or `.sops.yaml` | Encrypted files in repo |
The shared pre-commit hook (installed by `dev-env.gitHooks.enable`)
refuses to commit common secret patterns and unencrypted sops files.
See `dev-env/scripts/git-hooks/pre-commit` for the patterns.
## Local dev with `pass`
```bash
# pass is in dev-env's package set; gpg is in omni's defaults
gpg --gen-key # if you don't have one
pass init "your-email@example.com"
pass insert dev/lnbits/admin-key
pass insert dev/postgres/password
pass insert dev/bitcoin/rpc-password
pass dev/lnbits/admin-key # print
pass -c dev/lnbits/admin-key # to clipboard (clears after 45s)
export LNBITS_ADMIN_KEY=$(pass dev/lnbits/admin-key) # in scripts
```
## Server secrets with sops-nix
### Initial setup
```bash
age-keygen -o ~/.config/sops/age/keys.txt # generate your key
age-keygen -y ~/.config/sops/age/keys.txt # show your public key
# age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
### Per-project
In each deploy host directory:
```bash
cd ~/dev/deploy/unified/hosts/host5
cat > .sops.yaml << EOF
keys:
- &admin age1xxx... # you
- &host5 age1yyy... # the server's key
creation_rules:
- path_regex: secrets/host5\.yaml$
key_groups:
- age:
- *admin
- *host5
EOF
mkdir -p secrets
nvim secrets/host5.yaml # write the unencrypted file
sops -e -i secrets/host5.yaml # encrypt in place
```
The pre-commit hook will refuse to commit `secrets/host5.yaml` if it's
not encrypted. (False positive? `git commit --no-verify`.)
### Using in NixOS
```nix
{ config, pkgs, ... }:
{
imports = [ inputs.sops-nix.nixosModules.sops ];
sops.defaultSopsFile = ./secrets/host5.yaml;
sops.age.keyFile = "/var/lib/sops-nix/key.txt";
sops.secrets."lnbits/admin_key" = {};
sops.secrets."postgres/password" = {};
services.lnbits = {
adminKeyFile = config.sops.secrets."lnbits/admin_key".path;
};
}
```
### Bootstrapping a server's age key
On each NixOS host (one-time):
```bash
sudo mkdir -p /var/lib/sops-nix
sudo age-keygen -o /var/lib/sops-nix/key.txt
sudo chmod 600 /var/lib/sops-nix/key.txt
sudo age-keygen -y /var/lib/sops-nix/key.txt # add to .sops.yaml
```
## Best practices
1. **Never commit unencrypted secrets.** The pre-commit hook helps but
isn't a substitute for paying attention.
2. **Rotate after team changes** — especially when removing keys.
3. **Different secrets per environment.** staging ≠ production.
4. **Backup your master keys.** GPG and age private keys are the only
thing standing between you and a total loss.
5. **No secrets in `.devenv.conf` / `/etc/dev-env/config.sh`.** Those
files are world-readable.

286
modules/dev-env/docs/stack-overview.md Normal file
View file

@ -0,0 +1,286 @@
# Aiolabs stack — high-level overview
Orientation map for new contributors. The substrate is **Omnixient** (this
repo): a NixOS desktop that declaratively materialises every project
worktree below via `modules/dev-env/` + the
[`aiolabs` preset](../presets/aiolabs.nix).
```mermaid
flowchart TB
subgraph DEV["Omnixient — NixOS dev environment (this repo)"]
DEVENV["modules/dev-env + aiolabs preset<br/>worktrees · tmux · git hooks<br/>Forgejo origin + GitHub upstream forks"]
end
subgraph DEPLOY["Deploy"]
DEPLOYFLAKE["server-deploy flake<br/>host1 · host5 · host7 · host8<br/>host4 · host3 · host6 · 484"]
end
subgraph FRONT["User-facing frontends"]
direction LR
BITSPIRE["bitSpire<br/>Bitcoin ATM frontend<br/>KYC-free · Nostr-native"]
WEBAPP["webapp / 'AIO'<br/>Vue 3 + TS · Electron + PWA<br/>wallet · events · Nostr"]
end
subgraph CORE["LNbits — Lightning backend (Python / FastAPI)"]
LNBITS["lnbits core<br/>wallets · per-wallet API keys · extension runtime<br/>nostr-native transport (kind-21000, NIP-44 v2)"]
end
subgraph EXT["LNbits extensions — shared/extensions/"]
direction LR
SPIRE["spirekeeper<br/>bitSpire operator dashboard"]
NREL["nostrrelay"]
NCLI["nostrclient"]
EVENTS["events"]
NMKT["nostrmarket"]
LIBRA["libra"]
LNURLP["lnurlp"]
REST["restaurant"]
end
RELAYS(("Nostr relays<br/>event bus"))
BTC["Bitcoin / Lightning<br/>LND · CLN · Phoenixd · …"]
FAVA["Fava + Beancount<br/>double-entry ledger"]
REF["Lightning.Pub (shocknet)<br/>reference implementation for the<br/>nostr-native Lightning interface"]
DEV -. develops .-> CORE
DEV -. develops .-> FRONT
DEV -. drives .-> DEPLOY
DEPLOY -- ships --> CORE
DEPLOY -- ships --> FRONT
LNBITS -- hosts --> EXT
%% On Nostr there are no point-to-point links: the nostr-native
%% parts all meet on the relay bus, addressed by pubkey. bitSpire↔
%% LNbits (payments) and bitSpire↔spirekeeper (fleet) both resolve
%% through the relay, not over a direct edge.
BITSPIRE <-- "Lightning RPC + fleet (kind-21000)" --> RELAYS
SPIRE <-- "bitSpire fleet control" --> RELAYS
NCLI -- "LNbits ↔ relay" --> RELAYS
NREL -- "in-stack relay" --> RELAYS
EVENTS -- publishes --> RELAYS
NMKT -- marketplace --> RELAYS
WEBAPP -- subscribes --> RELAYS
%% non-Nostr edges
WEBAPP -- REST + WebSocket --> LNBITS
LNBITS -- funding source --> BTC
LIBRA -- HTTP / JSON API --> FAVA
REF -. reference impl .-> LNBITS
classDef reference stroke-dasharray: 4 4,opacity:0.65;
class REF reference;
```
## How to read this
- **Omnixient** is where you *work*. Cloning it and running the dev-env
bootstrap materialises every box below as a worktree under
`~/dev/`, set up with the right Forgejo origin and GitHub upstream
fork.
- **LNbits** is the backend platform. Everything user-visible either
*is* an LNbits extension or *talks to* LNbits. It holds wallets,
funds them from a Bitcoin/Lightning source, runs the extension
runtime, and exposes both a classic HTTP/WebSocket API and a
**nostr-native transport** — signed-event RPC over a relay
(kind-21000, NIP-44 v2) with no bearer tokens on the wire.
- **Extensions** are Python modules dropped into LNbits' extension
runtime. They're cloned independently into `shared/extensions/` so
multiple LNbits worktrees (dev/main) can share them.
- **libra** is the unusual one: it doesn't reimplement double-entry
accounting, it speaks to **Fava**'s JSON API. Fava is the web UI on
top of **Beancount** plain-text ledgers. So libra = LNbits plugin
that delegates accounting to a separate Fava service.
- **webapp** ("AIO") is the polished, abstracted user frontend. It
treats LNbits as a backend; configured via `VITE_LNBITS_BASE_URL`.
- **bitSpire** is the Bitcoin ATM frontend — a KYC-free, Nostr-native
ATM. It talks to its Lightning backend over the **nostr-native
transport** on a relay rather than HTTP, so the kiosk holds no admin
tokens: the ATM's own Nostr key *is* its credential, and LNbits
auto-provisions its wallet on first contact from that signature.
- **spirekeeper** is the operator control dashboard for bitSpire
fleets, built as an LNbits extension. It owns the operator side of a
deployment: fleet and cassette-inventory configuration, commission /
DCA distribution (parsed from each cash-out's `Payment.extra`),
operator branding, and telemetry — all carried over Nostr.
spirekeeper and bitSpire never connect directly: like every
nostr-native pair in the stack, they meet on the relay, each
addressed by its own Nostr pubkey.
- **server-deploy** is a separate flake that pins each project's
source revision and builds NixOS configurations for the production
hosts (`host8`, `host7`, `host5`, …).
> **Credit — Lightning.Pub.** LNbits' ATM-facing side — its
> nostr-native Lightning node-management interface — is modeled on
> [**Lightning.Pub**](https://github.com/shocknet/Lightning.Pub)
> (shocknet), the reference implementation for this pattern. bitSpire
> talks to LNbits the way it would talk to a Lightning.Pub node; the
> functionality lives in LNbits, but the shape of the interface is
> Lightning.Pub's.
## Future direction: Nostr-native transport
The `bitSpire ↔ LNbits` edge already runs on the nostr-native
transport — signed-event RPC over a relay, with Nostr's
key/signature model as the security primitive (auth, capabilities,
audit trail) instead of bearer tokens and TLS-only trust. The
longer-term direction is to move the *rest* of the stack's
inter-service edges the same way, so `webapp ↔ LNbits` and
`libra ↔ Fava` migrate from REST/WebSocket onto signed Nostr events
too. The proposed security structure is described in
[aiolabs/lnbits#9](https://git.atitlan.io/aiolabs/lnbits/issues/9).
The nostrclient/nostrrelay extensions already make LNbits a
first-class Nostr participant; the issue tracks the broader
auth/permissions design that has to land before the rest of the stack
can follow.
The same model is the candidate for **unified authentication**
across the whole platform: instead of each service running its own
account system (LNbits accounts, Authentik SSO, Forgejo logins, …),
a user's Nostr keypair becomes the identity and per-service
capabilities are granted as signed permission events. One identity,
one signing key, role-based access fanned out across LNbits,
Authentik, Forgejo and every other internal service.
## Deployment model: location-portable instances
The stack is designed to be **deployed anywhere** — every venue (a
co-living house, a makerspace, a community space, a bar running an
ATM) runs its own full instance: LNbits + the relevant extensions +
a local Nostr relay + supporting services. Each location is its own
sovereign tenant; nothing has to phone home to a central server.
Because identity lives in the user's Nostr keypair (see above)
rather than in any per-location account, the user experience is
*roaming*: when someone walks into a new location they point their
client at that location's relay and the same app surface comes up,
populated with that location's data — their local membership,
balances, ledger position, events. Identity travels with the user;
data stays scoped to the place.
The `server-deploy` flake already supports this shape — `host8`,
`host7`, `host4`, `host5` etc. are independent instances built
from the same source tree.
## Target architecture (Nostr-native, location-portable)
If everything above lands, the picture collapses dramatically. There
are no per-edge transports to draw and no per-service auth boxes —
just **one identity**, **one client surface**, and **N self-similar
location instances** that the client roams between by switching
relays.
```mermaid
flowchart TB
subgraph USER["The user"]
KEY["🔑 Nostr keypair<br/>identity · auth · permissions"]
CLIENT["Client surface<br/>webapp · bitSpire · future apps"]
KEY -. signs every event .-> CLIENT
end
subgraph LOC_A["Location A — e.g. Castle"]
RELAY_A(("Nostr relay"))
SVCS_A["LNbits + extensions<br/>Fava / Beancount<br/>Authentik · Forgejo<br/>Bitcoin / Lightning"]
SVCS_A <-- Nostr events --> RELAY_A
end
subgraph LOC_B["Location B"]
RELAY_B(("Nostr relay"))
SVCS_B["same stack<br/>per-location data"]
SVCS_B <-- Nostr events --> RELAY_B
end
subgraph LOC_N["Location N …"]
RELAY_N(("Nostr relay"))
SVCS_N["same stack<br/>per-location data"]
SVCS_N <-- Nostr events --> RELAY_N
end
CLIENT == currently connected ==> RELAY_A
CLIENT -. roam .-> RELAY_B
CLIENT -. roam .-> RELAY_N
```
Read this against the first diagram and the simplification is the
point: the tangle of REST/WebSocket arrows becomes a single
event-bus edge per service, the auth boxes disappear into the
keypair, and every location is the same shape. New locations are
just another copy of the bubble; new client apps are just another
signer using the same key.
## The ATM as cash ↔ sats bridge
The picture above is the *sovereign* half of the story. The other
half is that people still live in a **cash** economy — and at each
location the ATM (`bitSpire`) is the protected portal between the two
worlds. It sits inside the location instance and acts as a one-stop,
identity-preserving bridge:
- **Cash stays anonymous.** Walking up to the ATM with a banknote
needs no account, no KYC, no Nostr key. Cash keeps its
permissionless, locally-spendable nature — useful at every local
merchant whether or not they speak Lightning.
- **Onramp: cash → sats.** The ATM credits the user's LNbits wallet
at this location. The wallet is **custodial for now** — the
operator holds the sats, the way a bartender holds a cash tab. The
user trusts the operator to "carry the cash" between visits; the
trust is bounded by the escape hatch below.
- **Offramp: sats → cash.** The user can spend that location-wallet
balance back out as cash through the same ATM, or use it directly
with local merchants and LNbits extensions at the venue. No KYC
is required in either direction.
- **Escape to self-custody.** The custodial wallet is a checkpoint,
not a destination. One tap pulls sats out to the user's **cold
on-chain Bitcoin wallet** or to their **own self-custodied
Lightning wallet** — so anyone uncomfortable trusting the operator
can leave at any time with their full balance.
```mermaid
flowchart LR
subgraph FIAT["Local fiat economy"]
CASH["💵 Cash<br/>anonymous · permissionless<br/>spendable at local merchants"]
end
subgraph LOC["Location instance"]
direction TB
ATM["🏧 bitSpire ATM<br/>protected portal<br/>no KYC"]
WALLET["⚡ LNbits wallet<br/>custodial · operator-held<br/>Nostr key = login"]
STACK["LNbits + extensions<br/>local Nostr relay"]
ATM <-- "nostr-native transport" --> WALLET
WALLET --- STACK
end
subgraph SELF["User self-custody (off-platform)"]
COLD["❄️ cold on-chain<br/>Bitcoin wallet"]
OWNLN["⚡ own Lightning<br/>wallet / node"]
end
CASH <-- "cash ↔ sats" --> ATM
WALLET == "withdraw on-chain" ==> COLD
WALLET == "withdraw via LN" ==> OWNLN
```
The ATM is therefore the **gateway** that lets a user move value
freely between each location's legacy cash economy and the
Nostr-native ecosystem — without surrendering the anonymity that
cash and a Nostr keypair each provide on their own side of the
boundary. Cash works as cash; sats work as sats; the ATM is the
seam, and crossing it doesn't cost you your identity.
## Where the source lives
| Box | Path under `~/dev/` | Origin (Forgejo) |
|----------------------|--------------------------------|------------------------------------|
| Omnixient | (this repo, `/etc/nixos`) | `aiolabs/omnixient` |
| LNbits | `lnbits/{dev,main}` | `aiolabs/lnbits` (no fork) |
| Extensions | `shared/extensions/<name>` | `aiolabs/<name>` |
| spirekeeper | `shared/extensions/spirekeeper`| `aiolabs/spirekeeper` |
| webapp | `webapp/` | `aiolabs/webapp` |
| bitSpire | `bitspire/bitspire/{main,dev}` | `aiolabs/bitspire` |
| atm-tui | `bitspire/atm-tui` | `aiolabs/atm-tui` |
| server-deploy | `deploy/server-deploy` | `aiolabs/server-deploy` |
| nips (reference) | `nostr-protocol/nips` | nostr-protocol/nips (read-only) |
| regtest | `local/docker/regtest` | `aiolabs/regtest` |
> Lightning.Pub is kept as a read-only reference mirror under
> `~/dev/refs/` (see `refs.toml`), not as a runtime component.

181
modules/dev-env/docs/upstream-prs.md Normal file
View file

@ -0,0 +1,181 @@
# Contributing to Upstream Projects
How to contribute fixes and features back to upstream projects (lnbits,
lamassu, nix-bitcoin, Lightning.Pub) using the worktree-based workflow
provided by `dev-env`.
## Prerequisites (one-time, per repo)
1. **Fork the upstream repo on GitHub.** Click "Fork" on the upstream's
GitHub page (e.g. github.com/lnbits/lnbits) to create your personal
copy.
2. **Set your GitHub username** in the dev-env config (or override
per-project):
```nix
dev-env.github.forkUser = "your-github-username";
```
3. **Re-run `dev-env-bootstrap`.** It adds the `github-fork` remote to
the bare repo derived from `git@github.com:<forkUser>/<repo>.git`.
Verify with:
```bash
git -C ~/dev/repos/lnbits.git remote -v
# origin forgejo@git.atitlan.io:aiolabs/lnbits.git (fetch)
# upstream https://github.com/lnbits/lnbits (fetch)
# github-fork git@github.com:your-github-username/lnbits.git (fetch)
```
## Quick reference
| Command | Alias | Description |
|---|---|---|
| `git-pr-branch <repo> <branch>` | `prb` | Create PR worktree from upstream/main |
| `git-pr-cleanup <repo> <branch>` | `prc` | Remove worktree after PR merges |
| `git-pr-list` | `prl` | List active PR worktrees |
| `prs` | — | `cd ~/dev/upstream-prs` |
## Workflow
### 1. Create the PR branch
```bash
prb lnbits fix-invoice-validation
# Fetches upstream
# Creates 'fix-invoice-validation' from upstream/main
# Adds worktree at ~/dev/upstream-prs/lnbits-fix-invoice-validation
```
### 2. Make changes
```bash
cd ~/dev/upstream-prs/lnbits-fix-invoice-validation
git status # On branch fix-invoice-validation
nvim src/some_file.py
pytest tests/
git commit -am "Fix invoice validation for zero-amount invoices"
```
### 3. Push to your GitHub fork
```bash
git push github-fork fix-invoice-validation
```
### 4. Open the PR on GitHub
Go to upstream (e.g. github.com/lnbits/lnbits). You'll see the
"fix-invoice-validation had recent pushes" banner. Click "Compare &
pull request" and fill in title/description.
### 5. Address review feedback
```bash
cd ~/dev/upstream-prs/lnbits-fix-invoice-validation
nvim src/some_file.py
git commit -am "Address review: add input sanitization"
git push github-fork fix-invoice-validation
```
### 6. Cleanup after merge
```bash
prc lnbits fix-invoice-validation
# Removes worktree, deletes local branch
```
## Layout
```
~/dev/
├── repos/
│ └── lnbits.git # bare repo, three remotes:
│ ├── origin # forgejo (your team's fork)
│ ├── upstream # github (lnbits/lnbits)
│ └── github-fork # github (your-github-username/lnbits)
│
├── lnbits/ # team-fork worktrees
│ ├── dev/
│ └── main/
│
└── upstream-prs/
└── lnbits-fix-invoice-validation/ # transient PR worktree
```
## Git remotes explained
| Remote | Points to | Used for |
|---|---|---|
| `origin` | Forgejo | Your team's fork — `main`, `dev`, `feature/*` |
| `upstream` | GitHub (original) | Read-only; fetch latest upstream changes |
| `github-fork` | GitHub (your fork) | Write-only target for PR branches |
## Common scenarios
### Sync with upstream before starting
`prb` does this for you, but if you need to manually:
```bash
cd ~/dev/repos/lnbits.git
git fetch upstream
```
### Rebase on latest upstream mid-PR
```bash
cd ~/dev/upstream-prs/lnbits-fix-invoice-validation
git fetch upstream
git rebase upstream/main
git push github-fork fix-invoice-validation --force-with-lease
```
### Multiple PRs for the same repo
```bash
prb lnbits fix-invoice-validation
prb lnbits add-webhook-support
prb lnbits update-deps
prl # list all
```
### Abandon a PR
```bash
prc lnbits fix-invoice-validation # same as cleanup
```
## Best practices
1. **One PR per feature/fix.** Keep PRs focused and reviewable.
2. **Branch from upstream/main.** `prb` enforces this.
3. **Clear commit messages.** What and why, not just what.
4. **Test before pushing.**
5. **Cleanup after merge.** Don't accumulate stale worktrees.
6. **Rebase, not merge.** Keeps a clean history.
## Troubleshooting
**"github-fork remote not found"**
```bash
git -C ~/dev/repos/lnbits.git remote add github-fork \
git@github.com:$GITHUB_FORK_USER/lnbits.git
```
Or re-run `dev-env-bootstrap` after setting `dev-env.github.forkUser`.
**"Permission denied" pushing to github-fork**
- Make sure you've forked the repo on GitHub.
- Check `ssh -T git@github.com` succeeds.
- Verify the URL: `git -C ~/dev/repos/lnbits.git remote -v`.
**"Branch already exists"**
```bash
prc lnbits fix-x # cleanup first
prb lnbits fix-x # then recreate
```