aiolabs/spirekeeper
8
0
Fork 0
Code Issues 23 Pull requests Projects Releases Packages Wiki Activity Actions

chore(v2): collapse m001-m007 into single m001_satmachine_v2_initial

Browse source 
User confirmed no production servers are affected, so squashing the
staged migrations into a single source-of-truth migration is safe.

Reductions:
  m001-m004: legacy Lamassu schema (single-config + SSH-tunnel poller)
  m005:      v2 initial schema (had a SQLite CREATE-INDEX syntax bug)
  m006:      notes column
  m007:      processing_claim column + dca_machines.wallet_id UNIQUE
  ─────────  → m001_satmachine_v2_initial (single function)

What this commit changes:
  - Replaces seven migration functions with one. Diff -180 lines net
    (477 → 297). The collapsed migration carries the corrected SQLite
    syntax (no schema prefix on CREATE INDEX tables) and is idempotent
    end-to-end (CREATE TABLE/INDEX IF NOT EXISTS, seed check-then-insert).
  - All design choices the staged migrations earned are preserved in
    the inline comments: payment_hash idempotency key, absolute
    platform_fee_sats/operator_fee_sats, wallet_id UNIQUE defence-in-
    depth against IDOR, processing_claim optimistic-lock, notes
    append-only audit memo.
  - Pre-collapse history available in git on commits before this one.

What this commit does NOT change: schema. The final v2 tables / columns
/ indexes are identical to what m005+m006+m007 produced.

Upgrade path: anyone on the v2-bitspire branch with a partial-run
tracker (5/6/7) needs to uninstall + reinstall the extension to wipe
the dbversions tracker, then m001 runs fresh. Anyone on the legacy v1
main branch (tracker=4) does the same — uninstall + reinstall.

Refs: aiolabs/satmachineadmin#11 — closes the migration-collapse
follow-up that was deferred from fix bundle 3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Padreug 2026-05-14 19:15:28 +02:00
commit 2886dd7394
1 changed files with 90 additions and 277 deletions
Download patch file Download diff file Expand all files Collapse all files

363
migrations.py
View file

@ -1,192 +1,54 @@
# DCA Admin Extension Database Migrations
# Creates all necessary tables for Dollar Cost Averaging administration
# with Lamassu ATM integration
# Satoshi Machine v2 — single squashed migration.
#
# History note: m001-m004 were the legacy Lamassu schema; m005-m007 staged
# the v2 redesign (initial schema → payment_hash idempotency fix → notes
# column → concurrency claim + wallet UNIQUE index). Collapsed back into a
# single m001 during the v2-bitspire development branch since no production
# data was affected and the staged sequence had a SQLite CREATE-INDEX
# syntax bug. The pre-collapse history is preserved in git on commits
# prior to the collapse.
#
# Installs upgrading from the v1 Lamassu schema must uninstall + reinstall
# the extension to reset the LNbits dbversions tracker. The DROP TABLE
# IF EXISTS at the top of m001 also cleans the v1 tables if they happen
# to survive a partial wipe.
async def m001_initial_dca_schema(db):
async def m001_satmachine_v2_initial(db):
"""Single-shot v2 schema for the Satoshi Machine admin extension.
Drops every legacy Lamassu table (lamassu_config, lamassu_transactions,
plus the singular-config v1 dca_clients/deposits/payments) and creates
the v2 multi-tenant schema:
- super_config: singleton platform-fee config (super only)
- dca_machines: per-operator multi-machine registry by npub
- dca_clients: LP registrations scoped per (machine, user)
- dca_deposits: fiat the operator records against an LP
- dca_settlements: bitSpire kind-21000 idempotency table
- dca_commission_splits: operator's remainder-distribution rules
- dca_payments: leg-typed distribution audit trail
- dca_telemetry: sparse kind-30078/30079 snapshots per machine
CRITICAL design choices (preserved from the staged migrations):
* payment_hash is the UNIQUE idempotency key on dca_settlements
(LN payment_hash is globally unique and always present at the
Payment layer — fix for the original "use bitspire_event_id"
false start).
* platform_fee_sats + operator_fee_sats stored as absolute BIGINT
(not derived percentages). The contract is locked at landing time;
post-v1 customer-discount engine writes here without a migration.
* dca_machines.wallet_id UNIQUE — defence-in-depth against the
wallet-IDOR funds-theft vector (the API layer also checks
wallet ownership; the index is the second line of defence).
* processing_claim on dca_settlements — optimistic-lock token for
concurrent process_settlement invocations.
* notes on dca_settlements — append-only audit memo for partial-
dispense recompute + operator-authored notes (see
aiolabs/satmachineadmin#10 for the future structured audit table).
"""
Create complete DCA admin schema from scratch.
"""
# DCA Clients table
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_clients (
id TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
wallet_id TEXT NOT NULL,
username TEXT,
dca_mode TEXT NOT NULL DEFAULT 'flow',
fixed_mode_daily_limit INTEGER,
status TEXT NOT NULL DEFAULT 'active',
created_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now},
updated_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now}
);
"""
)
# DCA Deposits table
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_deposits (
id TEXT PRIMARY KEY NOT NULL,
client_id TEXT NOT NULL,
amount INTEGER NOT NULL,
currency TEXT NOT NULL DEFAULT 'GTQ',
status TEXT NOT NULL DEFAULT 'pending',
notes TEXT,
created_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now},
confirmed_at TIMESTAMP
);
"""
)
# DCA Payments table
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_payments (
id TEXT PRIMARY KEY NOT NULL,
client_id TEXT NOT NULL,
amount_sats INTEGER NOT NULL,
amount_fiat INTEGER NOT NULL,
exchange_rate REAL NOT NULL,
transaction_type TEXT NOT NULL,
lamassu_transaction_id TEXT,
payment_hash TEXT,
status TEXT NOT NULL DEFAULT 'pending',
created_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now}
);
"""
)
# Lamassu Configuration table
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.lamassu_config (
id TEXT PRIMARY KEY NOT NULL,
host TEXT NOT NULL,
port INTEGER NOT NULL DEFAULT 5432,
database_name TEXT NOT NULL,
username TEXT NOT NULL,
password TEXT NOT NULL,
source_wallet_id TEXT,
commission_wallet_id TEXT,
is_active BOOLEAN NOT NULL DEFAULT true,
test_connection_last TIMESTAMP,
test_connection_success BOOLEAN,
last_poll_time TIMESTAMP,
last_successful_poll TIMESTAMP,
use_ssh_tunnel BOOLEAN NOT NULL DEFAULT false,
ssh_host TEXT,
ssh_port INTEGER NOT NULL DEFAULT 22,
ssh_username TEXT,
ssh_password TEXT,
ssh_private_key TEXT,
created_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now},
updated_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now}
);
"""
)
# Lamassu Transactions table (for audit trail)
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.lamassu_transactions (
id TEXT PRIMARY KEY NOT NULL,
lamassu_transaction_id TEXT NOT NULL UNIQUE,
fiat_amount INTEGER NOT NULL,
crypto_amount INTEGER NOT NULL,
commission_percentage REAL NOT NULL,
discount REAL NOT NULL DEFAULT 0.0,
effective_commission REAL NOT NULL,
commission_amount_sats INTEGER NOT NULL,
base_amount_sats INTEGER NOT NULL,
exchange_rate REAL NOT NULL,
crypto_code TEXT NOT NULL DEFAULT 'BTC',
fiat_code TEXT NOT NULL DEFAULT 'GTQ',
device_id TEXT,
transaction_time TIMESTAMP NOT NULL,
processed_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now},
clients_count INTEGER NOT NULL DEFAULT 0,
distributions_total_sats INTEGER NOT NULL DEFAULT 0
);
"""
)
async def m002_add_transaction_time_to_dca_payments(db):
"""
Add transaction_time field to dca_payments table to store original ATM transaction time
"""
await db.execute(
"""
ALTER TABLE satoshimachine.dca_payments
ADD COLUMN transaction_time TIMESTAMP
"""
)
async def m003_add_max_daily_limit_config(db):
"""
Add max_daily_limit_gtq field to lamassu_config table for admin-configurable client limits
"""
await db.execute(
"""
ALTER TABLE satoshimachine.lamassu_config
ADD COLUMN max_daily_limit_gtq INTEGER NOT NULL DEFAULT 2000
"""
)
async def m004_convert_to_gtq_storage(db):
"""
Convert centavo storage to GTQ storage by changing data types and converting existing data.
Handles both SQLite (data conversion only) and PostgreSQL (data + schema changes).
"""
# Detect database type
db_type = str(type(db)).lower()
is_postgres = 'postgres' in db_type or 'asyncpg' in db_type
if is_postgres:
# PostgreSQL: Need to change column types first, then convert data
# Change column types to DECIMAL(10,2)
await db.execute("ALTER TABLE satoshimachine.dca_deposits ALTER COLUMN amount TYPE DECIMAL(10,2)")
await db.execute("ALTER TABLE satoshimachine.dca_payments ALTER COLUMN amount_fiat TYPE DECIMAL(10,2)")
await db.execute("ALTER TABLE satoshimachine.lamassu_transactions ALTER COLUMN fiat_amount TYPE DECIMAL(10,2)")
await db.execute("ALTER TABLE satoshimachine.dca_clients ALTER COLUMN fixed_mode_daily_limit TYPE DECIMAL(10,2)")
await db.execute("ALTER TABLE satoshimachine.lamassu_config ALTER COLUMN max_daily_limit_gtq TYPE DECIMAL(10,2)")
# Convert data from centavos to GTQ
await db.execute("UPDATE satoshimachine.dca_deposits SET amount = amount / 100.0 WHERE currency = 'GTQ'")
await db.execute("UPDATE satoshimachine.dca_payments SET amount_fiat = amount_fiat / 100.0")
await db.execute("UPDATE satoshimachine.lamassu_transactions SET fiat_amount = fiat_amount / 100.0")
await db.execute("UPDATE satoshimachine.dca_clients SET fixed_mode_daily_limit = fixed_mode_daily_limit / 100.0 WHERE fixed_mode_daily_limit IS NOT NULL")
await db.execute("UPDATE satoshimachine.lamassu_config SET max_daily_limit_gtq = max_daily_limit_gtq / 100.0 WHERE max_daily_limit_gtq > 1000")
else:
# SQLite: Data conversion only (dynamic typing handles the rest)
await db.execute("UPDATE satoshimachine.dca_deposits SET amount = CAST(amount AS REAL) / 100.0 WHERE currency = 'GTQ'")
await db.execute("UPDATE satoshimachine.dca_payments SET amount_fiat = CAST(amount_fiat AS REAL) / 100.0")
await db.execute("UPDATE satoshimachine.lamassu_transactions SET fiat_amount = CAST(fiat_amount AS REAL) / 100.0")
await db.execute("UPDATE satoshimachine.dca_clients SET fixed_mode_daily_limit = CAST(fixed_mode_daily_limit AS REAL) / 100.0 WHERE fixed_mode_daily_limit IS NOT NULL")
await db.execute("UPDATE satoshimachine.lamassu_config SET max_daily_limit_gtq = CAST(max_daily_limit_gtq AS REAL) / 100.0 WHERE max_daily_limit_gtq > 1000")
async def m005_satmachine_v2_overhaul(db):
"""
BREAKING REDESIGN — Satoshi Machine v2 (bitSpire integration + multi-tenant).
Drops the v1 Lamassu-era tables (SSH/PostgreSQL polling, single-config, super-only)
and creates the v2 schema for:
- Per-operator multi-machine support (1 LNbits user = 1 operator, N machines).
- bitSpire (Nostr kind-21000) settlement subscription instead of SQL polling.
- Two-stage commission split (platform fee first, operator-defined remainder).
- Absolute platform_fee_sats / operator_fee_sats storage on settlements (v1 hook
for v2 customer-discount engine — see plan section "Customer discounts").
Operators on the previous schema must wipe & re-onboard. No backwards-compat.
"""
# Drop v1 tables. IF EXISTS is safe both on upgrade and fresh-install paths.
# 1. Drop legacy v1 tables. IF EXISTS handles both fresh-install
# paths (no-op) and migration from a v1 schema (cleans up).
for table in (
"lamassu_transactions",
"lamassu_config",
@ -196,8 +58,7 @@ async def m005_satmachine_v2_overhaul(db):
):
await db.execute(f"DROP TABLE IF EXISTS satoshimachine.{table}")
# super_config — singleton (id='default') holding super's platform-fee config.
# The only thing the LNbits super has direct DB control over in this extension.
# 2. super_config — singleton (id='default') with platform-fee config.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.super_config (
@ -208,8 +69,6 @@ async def m005_satmachine_v2_overhaul(db):
);
"""
)
# Idempotent seed: check before insert so re-runs after a partial-
# failure recovery don't trip the PK conflict.
existing = await db.fetchone(
"SELECT id FROM satoshimachine.super_config WHERE id = 'default'"
)
@ -219,9 +78,8 @@ async def m005_satmachine_v2_overhaul(db):
"VALUES ('default', 0.0000)"
)
# dca_machines — one row per bitSpire ATM, owned by exactly one operator.
# fallback_commission_pct kicks in only when bitSpire's settlement Payment.extra
# is missing the (net_sats, fee_sats) split — see plan's lamassu-next ask #1.
# 3. dca_machines — one row per bitSpire ATM, owned by exactly one
# operator. wallet_id UNIQUE prevents the IDOR funds-theft vector.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_machines (
@ -243,9 +101,14 @@ async def m005_satmachine_v2_overhaul(db):
"CREATE INDEX IF NOT EXISTS dca_machines_operator_idx "
"ON dca_machines (operator_user_id)"
)
await db.execute(
"CREATE UNIQUE INDEX IF NOT EXISTS dca_machines_wallet_id_uq "
"ON dca_machines (wallet_id)"
)
# dca_clients — LP registrations scoped per (machine, user). One LP can hold
# positions across many machines (and many operators) on the same instance.
# 4. dca_clients — LP registrations scoped per (machine, user). An LP
# can hold positions at many machines (and many operators) on the
# same LNbits instance.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_clients (
@ -273,8 +136,8 @@ async def m005_satmachine_v2_overhaul(db):
"ON dca_clients (user_id)"
)
# dca_deposits — fiat the operator (or super) records against an LP at a machine.
# creator_user_id preserves audit trail (resolves a v1 tech-debt finding).
# 5. dca_deposits — fiat the operator (or super) records against an LP
# at a machine. creator_user_id preserves audit trail.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_deposits (
@ -296,20 +159,17 @@ async def m005_satmachine_v2_overhaul(db):
"ON dca_deposits (client_id, created_at DESC)"
)
# dca_settlements — idempotency table for bitSpire-driven settlements.
# The natural unique key is payment_hash (every LN invoice has a globally
# unique hash; subscription replays / dispatcher double-fires dedup via the
# UNIQUE constraint). bitspire_event_id is reserved for a future path where
# we subscribe to raw Nostr events directly (kind-30078/30079 ingestion
# uses dca_telemetry; bitspire_event_id is kept here for future bookkeeping
# if we ever bypass the LNbits Payment system).
# 6. dca_settlements — idempotency table for bitSpire-driven settlements.
# payment_hash UNIQUE handles subscription replays + dispatcher
# double-fires. processing_claim is the optimistic-lock token
# written by claim_settlement_for_processing. notes is the
# append-only audit memo for partial-dispense + operator notes.
#
# CRITICAL: platform_fee_sats and operator_fee_sats are stored as absolute
# BIGINT (not a derived percentage). Today this is just the contractual
# split. Once the v2 promotion engine ships, the two values diverge when
# discounts fire and this row is the only audit-grade record of who forgave
# what. Do not collapse them into a single commission_pct field. See plan
# section "Customer discounts".
# platform_fee_sats and operator_fee_sats are absolute BIGINT,
# NOT derived percentages — when the v2 customer-discount engine
# ships, these two columns are the audit-grade record of who
# forgave what per transaction. Do not collapse them into a single
# commission_pct. See plan section "Customer discounts" and #10.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_settlements (
@ -333,7 +193,9 @@ async def m005_satmachine_v2_overhaul(db):
status TEXT NOT NULL DEFAULT 'pending',
error_message TEXT,
processed_at TIMESTAMP,
created_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now}
created_at TIMESTAMP NOT NULL DEFAULT {db.timestamp_now},
notes TEXT,
processing_claim TEXT
);
"""
)
@ -341,13 +203,12 @@ async def m005_satmachine_v2_overhaul(db):
"CREATE INDEX IF NOT EXISTS dca_settlements_machine_idx "
"ON dca_settlements (machine_id, created_at DESC)"
)
# payment_hash UNIQUE already creates a lookup index — no extra index needed.
# dca_commission_splits — operator's rules for distributing the *remainder*
# of each commission (commission_sats - platform_fee_sats). One row per leg.
# machine_id=NULL means "operator's default rules"; non-null means per-machine
# override. Sum of pct across rows for a given (machine_id, operator_user_id)
# scope must equal 1.0 — enforced at write-time in crud.py.
# 7. dca_commission_splits — operator's rules for distributing the
# *remainder* (commission_sats - platform_fee_sats). One row per
# leg. machine_id=NULL = operator default; non-null = per-machine
# override. Sum(pct) per (operator, machine) must equal 1.0 —
# enforced at write-time in crud.py.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_commission_splits (
@ -367,10 +228,10 @@ async def m005_satmachine_v2_overhaul(db):
"ON dca_commission_splits (operator_user_id, machine_id)"
)
# dca_payments — every leg of every distribution. The leg_type discriminator
# tells the audit story: dca | super_fee | operator_split | settlement (= the
# "settle small remainder at current rate" feature, see satmachineadmin#4) |
# autoforward (see satmachineadmin#8) | refund.
# 8. dca_payments — every leg of every distribution. leg_type
# discriminator: dca | super_fee | operator_split | settlement |
# autoforward | refund. status enum: pending | completed | failed |
# voided | skipped | refunded.
await db.execute(
f"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_payments (
@ -406,12 +267,12 @@ async def m005_satmachine_v2_overhaul(db):
"ON dca_payments (operator_user_id, leg_type)"
)
# dca_telemetry — latest replaceable kind-30078 (public availability beacon)
# and kind-30079 (operator-only fleet telemetry) snapshots per machine. The
# beacon today (lamassu-next/dev @ 2b712af) ships only cash_in/cash_out/
# cash_level/fiat/model — the post-#43 fields (name, location, geo, fees,
# limits, denominations, version) are nullable until that upstream issue
# lands. Ingest opportunistically; render absent fields gracefully in the UI.
# 9. dca_telemetry — latest replaceable kind-30078 (public availability
# beacon) and kind-30079 (operator-only fleet telemetry) snapshots
# per machine. The beacon today (lamassu-next/dev @ 2b712af) ships
# only cash_in/cash_out/cash_level/fiat/model — post-#43 fields
# (name, location, geo, fees, limits, denominations, version) are
# nullable until that upstream issue lands. Ingest opportunistically.
await db.execute(
"""
CREATE TABLE IF NOT EXISTS satoshimachine.dca_telemetry (
@ -434,51 +295,3 @@ async def m005_satmachine_v2_overhaul(db):
);
"""
)
async def m006_add_settlement_notes(db):
"""Audit memo on dca_settlements.
When an operator triggers an in-place adjustment (partial-dispense,
manual reconciliation override, etc.), the settlement row's monetary
fields are overwritten with the new numbers. To preserve the audit
trail without a separate history table, we append a timestamped memo
to this notes column capturing the previous values and the reason.
Operators see this directly in the settlement detail view, so any
overwrite is visible and dated. Append-only convention: new memos
are prepended with a timestamp; never edited in place.
"""
await db.execute(
"ALTER TABLE satoshimachine.dca_settlements ADD COLUMN notes TEXT"
)
async def m007_settlement_claim_and_machine_wallet_unique(db):
"""Security + concurrency hardening (fix bundle 1).
1. Adds `processing_claim` to dca_settlements. The settlement processor
uses an optimistic-lock pattern: write a per-invocation claim token
alongside the status='processing' flip, then re-read and confirm the
persisted token matches. Two concurrent process_settlement invocations
on the same id can't both win the claim, so no duplicate leg
creation / double-pay.
2. Adds a UNIQUE index on dca_machines.wallet_id so two machine rows
can never claim the same wallet. Closes a wallet-IDOR funds-theft
vector where operator A could register a machine on operator B's
wallet_id and drain it via the settlement processor's pay_invoice.
Defence-in-depth on top of the API-layer ownership check; if a future
endpoint forgets the check, the DB still rejects.
CREATE UNIQUE INDEX is portable across SQLite and PostgreSQL
(ALTER TABLE ADD CONSTRAINT is not on SQLite).
"""
await db.execute(
"ALTER TABLE satoshimachine.dca_settlements "
"ADD COLUMN processing_claim TEXT"
)
await db.execute(
"CREATE UNIQUE INDEX IF NOT EXISTS dca_machines_wallet_id_uq "
"ON dca_machines (wallet_id)"
)