diff --git a/README.md b/README.md index 183ee5e..dbac287 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,6 @@ top left corner of the documents. * [Configuration and maintenance](docs/configuration.md) * [Using services](docs/services.md) * [FAQ](docs/faq.md) -* [Security model](docs/security-model.md) Features --- diff --git a/docs/security-model.md b/docs/security-model.md deleted file mode 100644 index dd4a91b..0000000 --- a/docs/security-model.md +++ /dev/null @@ -1,358 +0,0 @@ -# Security Model - -This document explains how nix-bitcoin protects your node and funds. It is intended -for operators who may not be familiar with NixOS security primitives. - -For vulnerability reporting and the security fund, see [SECURITY.md](../SECURITY.md). - ---- - -## Table of Contents - -- [Secrets Management](#secrets-management) - - [What is stored](#what-is-stored) - - [When secrets are generated](#when-secrets-are-generated) - - [File permissions](#file-permissions) - - [What is NOT in the secrets directory](#what-is-not-in-the-secrets-directory) -- [Wallet Key Material](#wallet-key-material) - - [LND](#lnd) - - [bitcoind](#bitcoind) -- [Service Isolation](#service-isolation) - - [Unix users and groups](#unix-users-and-groups) - - [systemd hardening](#systemd-hardening) - - [Network namespace isolation](#network-namespace-isolation) - - [RPC whitelisting](#rpc-whitelisting) - - [Operator user](#operator-user) -- [Network Security](#network-security) - - [RPC binding](#rpc-binding) - - [Tor](#tor) - - [WireGuard](#wireguard) - - [Firewall and D-Bus](#firewall-and-d-bus) -- [Hardening Presets](#hardening-presets) -- [Threat Model Summary](#threat-model-summary) -- [Operator Responsibilities](#operator-responsibilities) -- [Further Reading](#further-reading) - ---- - -## Secrets Management - -nix-bitcoin has its own secrets system for managing service credentials. Secrets -are files stored in a dedicated directory on the node, protected by unix -permissions and never written to the -[Nix store](https://nixos.org/manual/nix/stable/store/) (which is world-readable). - -The secrets directory location depends on the deployment method: -- **Krops / NixOps:** `/var/src/secrets` -- **Flakes / Containers:** `/etc/nix-bitcoin-secrets` - -### What is stored - -Each enabled service registers the secret files it needs. The secrets fall into -these categories: - -**Passwords** (random 20-character strings): -- `bitcoin-rpcpassword-privileged`, `bitcoin-rpcpassword-public` — bitcoind RPC authentication -- `lnd-wallet-password` — encrypts LND's `wallet.db` on disk -- `rtl-password`, `joinmarket-password`, `btcpayserver-password`, `lightning-loop-password` — web UI and service passwords -- `backup-encryption-env` — backup encryption passphrase - -**Derived credentials** (computed from passwords): -- `bitcoin-HMAC-privileged`, `bitcoin-HMAC-public` — HMAC hashes used in bitcoind's `rpcauth=` config -- Same pattern for Liquid (`liquid-rpcpassword-*`, `liquid-HMAC-*`) - -**TLS keys and certificates:** -- `lnd-key` — EC private key (prime256v1) for LND's gRPC/REST API -- `lnd-cert` — self-signed x509 certificate (10-year validity) - -**Other keys:** -- `clightning-replication-ssh-key` — for database replication -- WireGuard server/peer private and public keys (when the WireGuard preset is enabled) - -### When secrets are generated - -Secrets generation depends on the deployment method: - -| Method | Where generated | Mechanism | -|--------|----------------|-----------| -| **Krops** | Locally (your machine) | `generate-secrets` shell command runs before rsync to target | -| **NixOps** | Locally (your machine) | `generate-secrets` shell command, transferred via NixOps keys | -| **Flakes** | On the target node | `setup-secrets.service` at boot with `nix-bitcoin.generateSecrets = true` | -| **Containers** | Inside the container | Same as Flakes | -| **Manual** | You create them yourself | Set `nix-bitcoin.secretsSetupMethod = "manual"` | - -For Krops and NixOps, secrets are generated once locally and then synced to the -target. They are idempotent: existing files are never overwritten. If you delete -a secret file and redeploy, it will be regenerated. - -### File permissions - -The `setup-secrets` systemd service enforces permissions on every boot or deploy: - -1. The secrets directory is set to `root:root 0700` during setup -2. Each secret file is assigned ownership and permissions as declared by its - module (e.g. `lnd-wallet-password` is owned by `lnd:lnd 0440`) -3. Any file in the directory not claimed by a module is locked to `root:root 0440` -4. The directory is opened to `0751` after setup completes, allowing services - to access their specific files - -All nix-bitcoin services declare a systemd dependency on `nix-bitcoin-secrets.target`, -which is only reached after `setup-secrets` completes successfully. - -### What is NOT in the secrets directory - -The secrets system manages **service-to-service credentials**. The following are -explicitly outside its scope: - -- **LND seed mnemonic** (`/var/lib/lnd/lnd-seed-mnemonic`) — see [Wallet Key Material](#wallet-key-material) -- **LND wallet database** (`/var/lib/lnd/chain/bitcoin//wallet.db`) -- **LND macaroons** (`/var/lib/lnd/chain/bitcoin//*.macaroon`) -- **LND channel backup** (`/var/lib/lnd/chain/bitcoin//channel.backup`) -- **bitcoind wallet** (`wallet.dat` or descriptors, if created by the operator) - ---- - -## Wallet Key Material - -### LND - -LND uses the [aezeed cipher seed](https://github.com/lightningnetwork/lnd/tree/master/aezeed) -scheme — a 24-word mnemonic that encodes the wallet's master entropy and a -birthday timestamp. - -**How the wallet is created on first boot:** - -The LND module's `preStart` script checks if `wallet.db` exists. If not: - -1. `lndinit gen-seed` generates a fresh 24-word aezeed mnemonic and writes it - to `/var/lib/lnd/lnd-seed-mnemonic` -2. `lndinit init-wallet` creates `wallet.db` using the seed and the - `lnd-wallet-password` from the secrets directory -3. LND starts and auto-unlocks using the wallet password file - -**Important details:** - -- **No cipher seed passphrase:** nix-bitcoin does not set an aezeed passphrase. - The mnemonic is encrypted with the default passphrase `"aezeed"`, which offers - no real protection. Anyone with the 24 words can derive all keys. -- **Auto-unlock tradeoff:** The wallet password is stored on disk in the secrets - directory so LND can start unattended. This means root access to the node - grants access to the wallet. -- **The seed file is only used once:** After `wallet.db` is created, LND never - reads the seed file again. It can and should be deleted from disk after backup. -- **Static Channel Backups (SCB):** LND maintains `channel.backup` which is - updated atomically every time a channel is opened or closed. It is encrypted - with a key derived from the seed. If you re-seed, old SCBs become invalid. - -### bitcoind - -nix-bitcoin does **not** create or manage a bitcoind wallet. The `bitcoind.nix` -module configures the daemon and RPC authentication only. If you create a wallet -via `bitcoin-cli createwallet`, its key material is entirely your responsibility -to manage and back up. - ---- - -## Service Isolation - -### Unix users and groups - -Each nix-bitcoin service runs as its own dedicated system user (e.g. `bitcoind`, -`lnd`, `clightning`). Services cannot read each other's data directories unless -explicitly granted access through group memberships. - -### systemd hardening - -All nix-bitcoin services apply a strict systemd security profile -([`pkgs/lib.nix`](../pkgs/lib.nix)), which includes: - -| Setting | Effect | -|---------|--------| -| `ProtectSystem = "strict"` | Filesystem is read-only except for explicitly allowed paths | -| `ProtectHome = true` | No access to home directories | -| `PrivateTmp = true` | Isolated `/tmp` per service | -| `PrivateDevices = true` | No access to physical devices | -| `NoNewPrivileges = true` | Cannot gain new privileges via setuid/setgid | -| `MemoryDenyWriteExecute = true` | Prevents writable+executable memory (JIT disabled) | -| `ProtectKernelTunables = true` | Cannot modify kernel parameters | -| `ProtectKernelModules = true` | Cannot load kernel modules | -| `ProtectProc = "invisible"` | Other processes hidden in `/proc` | -| `PrivateUsers = true` | Cannot see other users | -| `IPAddressDeny = "any"` | All network denied by default (services opt in to what they need) | -| `CapabilityBoundingSet = ""` | No Linux capabilities | -| `SystemCallFilter` | Restricted to `@system-service` syscall set | -| `RestrictAddressFamilies` | Only `AF_UNIX`, `AF_INET`, `AF_INET6` | - -For the full list, see `man systemd.exec` and `man systemd.resource-control`. - -### Network namespace isolation - -The optional [`netns-isolation`](../modules/netns-isolation.nix) module places -each service in its own -[network namespace](https://man7.org/linux/man-pages/man7/network_namespaces.7.html). -Services can only communicate with other services through explicitly allowed -paths. For example, LND can reach bitcoind's RPC, but RTL cannot directly reach -bitcoind. - -Enable with: -```nix -nix-bitcoin.netns-isolation.enable = true; -``` - -Note: This is not compatible with the WireGuard preset. - -### RPC whitelisting - -The [`bitcoind-rpc-public-whitelist`](../modules/bitcoind-rpc-public-whitelist.nix) -module restricts which RPC methods the `public` RPC user can call. Services that -only need read access (like electrs or mempool) use the `public` user, which -cannot call wallet or administrative RPCs. Only the `privileged` user has full -RPC access. - -### Operator user - -The [`operator`](../modules/operator.nix) module creates a non-root user with -group memberships for each enabled service. This lets you run `bitcoin-cli`, -`lncli`, `lightning-cli`, etc. without being root. The operator user has read -access to service data but does not have write access or the ability to modify -service configuration. - ---- - -## Network Security - -### RPC binding - -bitcoind and LND bind their RPC/API interfaces to `127.0.0.1` by default. They -are not reachable from outside the machine unless you explicitly change the bind -address. - -### Tor - -The [`secure-node`](../modules/presets/secure-node.nix) preset imports -[`enable-tor`](../modules/presets/enable-tor.nix), which routes all outbound -traffic from nix-bitcoin services through Tor. Services that support it can also -accept inbound connections via -[onion services](https://community.torproject.org/onion-services/overview/). - -When Tor enforcement is active (`tor.enforce = true`), a service's systemd -`IPAddressAllow` is restricted to localhost and link-local addresses only, -preventing any clearnet communication. - -### WireGuard - -The [`wireguard`](../modules/presets/wireguard.nix) preset creates an encrypted -VPN tunnel for connecting a mobile wallet (e.g. Zeus) to your node. It sets up a -single-peer WireGuard interface with: - -- The node as server (`10.10.0.1`) -- Your device as peer (`10.10.0.2`) -- Firewall rules restricting the peer to only reach the node's address (no - routing to the broader network) -- Helper commands (`nix-bitcoin-wg-connect`, `lndconnect-wg`) that generate QR - codes for one-scan setup - -This is an alternative to connecting over Tor, offering lower latency at the -cost of requiring a reachable IP and port forwarding for NAT. - -### Firewall and D-Bus - -The `secure-node` preset enables the NixOS firewall and the -[`security`](../modules/security.nix) module's D-Bus process information hiding. -The D-Bus restriction prevents unprivileged services from discovering other -services' process information (command lines, cgroup paths) via systemd's D-Bus -interface. - ---- - -## Hardening Presets - -nix-bitcoin provides three presets that can be combined: - -**[`secure-node.nix`](../modules/presets/secure-node.nix)** — Opinionated base -configuration: -- Enables firewall -- Routes all traffic through Tor -- Replaces `sudo` with `doas` -- Enables D-Bus process information hiding -- Creates an SSH onion service -- Enables the operator user -- Sets up daily backups -- Enables `nodeinfo` command - -**[`hardened.nix`](../modules/presets/hardened.nix)** — Imports the -[NixOS hardened kernel profile](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/profiles/hardened.nix). -This enables kernel-level hardening (address space layout randomization, kernel -module restrictions, etc.) at a ~50% performance cost. Resets `allowUserNamespaces` -to `true` (needed for Nix sandboxing) and uses the standard `libc` allocator. - -**[`hardened-extended.nix`](../modules/presets/hardened-extended.nix)** — Builds -on `hardened.nix` with additional restrictions: -- Disables kernel log leaks, SysRq, debugfs -- Restricts ptrace, TTY line disciplines, core dumps -- Disables TCP SACK and timestamps -- Blacklists obscure network protocols, rare filesystems, Bluetooth, webcam, - Thunderbolt/FireWire (DMA attack prevention) -- Enables USBGuard -- Enforces signed kernel modules and kernel lockdown - -See [madaidan's Linux Hardening Guide](https://madaidans-insecurities.github.io/guides/linux-hardening.html) -for background on these settings. - -None of these presets affect secrets generation — that is controlled separately -by the deployment method and `nix-bitcoin.generateSecrets`. - ---- - -## Threat Model Summary - -| Scenario | Impact | Notes | -|----------|--------|-------| -| **Attacker has root on the node** | Full compromise. All funds at risk. | `lnd-wallet-password` is on disk, allowing wallet decryption. All secrets are readable. | -| **Secrets directory leaked, no network access** | No direct fund theft. | Secrets contain RPC passwords and TLS keys but not wallet key material. Without network access to the RPC port, passwords are not exploitable. | -| **Secrets directory leaked, with RPC network access** | bitcoind wallet funds at risk (if a wallet exists). | The `privileged` RPC password allows calling any bitcoind RPC, including spending. LND funds are safer: the attacker also needs a macaroon (not in secrets dir) to call LND RPCs. | -| **LND seed mnemonic leaked** | All LND on-chain funds compromised. | The attacker can derive all keys. Channel funds are at risk if the attacker broadcasts old commitment transactions. Immediate action required: sweep funds, close channels, re-seed. | -| **Nix store accessed** | No secrets exposed. | Secrets are never written to the Nix store. Configuration files reference secret file paths, not secret values. | -| **Deploy machine compromised** | Full compromise. | For Krops/NixOps, the deploy machine holds plaintext secrets and controls what code is deployed to the node. | - ---- - -## Operator Responsibilities - -These items are not automated by nix-bitcoin and require manual action: - -- [ ] **Back up the LND seed mnemonic.** After first boot, copy - `/var/lib/lnd/lnd-seed-mnemonic` to secure offline storage. Then delete the - file from the node. This is the only way to recover on-chain funds. - -- [ ] **Back up channel state.** Copy - `/var/lib/lnd/chain/bitcoin/mainnet/channel.backup` after opening new channels. - This allows off-chain fund recovery via the - [Data Loss Protection protocol](https://github.com/lightningnetwork/lnd/blob/master/docs/recovery.md). - Alternatively, enable `services.backups` to automate this. - -- [ ] **Secure the deploy machine.** For Krops and NixOps deployments, your local - `secrets/` directory contains plaintext credentials. If your local machine is - compromised, the node is compromised. - -- [ ] **Review enabled presets.** Consider enabling `secure-node.nix` (Tor, - firewall, operator user) and `netns-isolation` (network namespace separation - between services). These are not enabled by default. - -- [ ] **Understand the auto-unlock tradeoff.** LND's wallet password is stored - on disk so the service can start unattended. This means anyone with root access - to the node can unlock the wallet. There is no way to require manual unlock at - boot within the current nix-bitcoin design. - ---- - -## Further Reading - -- [NixOS Hardened Profile](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/profiles/hardened.nix) — kernel and userspace hardening applied by `hardened.nix` -- [systemd.exec(5)](https://www.freedesktop.org/software/systemd/man/systemd.exec.html) — reference for the systemd sandboxing options used in `pkgs/lib.nix` -- [systemd.resource-control(5)](https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html) — resource limiting and IP address filtering -- [Linux network namespaces](https://man7.org/linux/man-pages/man7/network_namespaces.7.html) — the kernel feature behind `netns-isolation` -- [aezeed cipher seed](https://github.com/lightningnetwork/lnd/tree/master/aezeed) — LND's seed scheme and how it differs from BIP39 -- [LND fund recovery](https://github.com/lightningnetwork/lnd/blob/master/docs/recovery.md) — on-chain and off-chain recovery procedures -- [madaidan's Linux Hardening Guide](https://madaidans-insecurities.github.io/guides/linux-hardening.html) — background for `hardened-extended.nix` settings -- [Nix Store security](https://nixos.org/manual/nix/stable/store/) — why secrets must not be written to the store diff --git a/modules/lamassu-lnbits.nix b/modules/lamassu-lnbits.nix index 84b58a2..a502fcc 100644 --- a/modules/lamassu-lnbits.nix +++ b/modules/lamassu-lnbits.nix @@ -7,65 +7,26 @@ let nbLib = config.nix-bitcoin.lib; secretsDir = config.nix-bitcoin.secretsDir; - # Shared environment variables for both services - commonEnv = { - NODE_ENV = cfg.mode; - LOG_LEVEL = cfg.logLevel; - HOSTNAME = cfg.hostname; + # Source directory for lamassu-server (cloned from git) + lamassuSourceDir = "${cfg.dataDir}/source"; - # Database - POSTGRES_HOST = "127.0.0.1"; - POSTGRES_PORT = "5432"; - POSTGRES_DB = cfg.database.name; - POSTGRES_USER = cfg.database.user; - - # TLS certificates - CA_PATH = cfg.certPath; - CERT_PATH = cfg.certPath; - KEY_PATH = cfg.keyPath; - - # Data directories - MNEMONIC_PATH = "${cfg.dataDir}/lamassu-mnemonic"; - OFAC_DATA_DIR = "${cfg.dataDir}/ofac"; - ID_PHOTO_CARD_DIR = "${cfg.dataDir}/photos/idcards"; - FRONT_CAMERA_DIR = "${cfg.dataDir}/photos/frontcamera"; - OPERATOR_DATA_DIR = "${cfg.dataDir}/operator"; - - # Security - SKIP_2FA = if cfg.skip2FA then "true" else "false"; - }; - - # Shared wrapper script that sets up the environment - lamassuEnv = pkgs.writeShellScript "lamassu-env" '' - set -euo pipefail - export PATH=${pkgs.nodejs_22}/bin:$PATH - DB_PASSWORD=$(cat ${secretsDir}/lamassu-db-password) - export DATABASE_URL="postgresql://${cfg.database.user}:$DB_PASSWORD@127.0.0.1:5432/${cfg.database.name}" - export POSTGRES_PASSWORD="$DB_PASSWORD" - export NODE_PATH=${cfg.dataDir}/source/node_modules - cd ${cfg.dataDir}/source - exec "$@" - ''; - - # Hardening settings for runtime services - hardeningConfig = { + # Basic hardening settings (simplified from nix-bitcoin) + defaultHardening = { + # Sandboxing PrivateTmp = true; ProtectSystem = "strict"; ProtectHome = true; NoNewPrivileges = true; + + # Kernel ProtectKernelTunables = true; ProtectKernelModules = true; ProtectControlGroups = true; + + # Misc RestrictRealtime = true; RestrictSUIDSGID = true; LockPersonality = true; - MemoryDenyWriteExecute = false; # Required for Node.js JIT - ReadWritePaths = [ cfg.dataDir ]; - RestrictAddressFamilies = [ "AF_UNIX" "AF_INET" "AF_INET6" ]; - User = cfg.user; - Group = cfg.group; - Restart = "on-failure"; - RestartSec = "10s"; }; in @@ -79,6 +40,13 @@ in description = "Port for the main lamassu server API"; }; + # NOTE: Admin UI port is currently hardcoded in upstream lamassu-server: + # - Production mode (default): port 443 + # - Dev mode (--dev flag): port 8070 + # Future: Add --ui-port support to upstream to make this configurable. + # This would also enable nginx reverse proxy (which also needs port 443). + # See docs/lamassu-future-nginx.md for implementation details. + dataDir = mkOption { type = types.path; default = "/var/lib/lamassu-server"; @@ -97,6 +65,12 @@ in description = "Group to run lamassu-server as"; }; + package = mkOption { + type = types.path; + default = lamassuSourceDir; + description = "The path to the lamassu-server source directory"; + }; + source = { url = mkOption { type = types.str; @@ -117,21 +91,20 @@ in description = "Logging level for lamassu-server"; }; - mode = mkOption { - type = types.enum [ "production" "development" ]; - default = "development"; - description = '' - Run in production or development mode. - Development mode uses port 3001 for admin UI registration URLs. - ''; - }; - skip2FA = mkOption { type = types.bool; - default = false; - description = "Skip 2FA authentication (only enable for initial setup, then disable)"; + default = true; + description = "Skip 2FA authentication (useful for initial setup)"; }; + # NOTE: devMode is disabled for now. Admin UI runs in production mode (port 443). + # Future: Re-enable when --ui-port is added to upstream lamassu-server. + # devMode = mkOption { + # type = types.bool; + # default = false; + # description = "Run admin server in development mode (port 8070)."; + # }; + database = { name = mkOption { type = types.str; @@ -144,6 +117,9 @@ in default = cfg.user; description = "PostgreSQL username"; }; + + # Password is managed by nix-bitcoin secrets system. + # See: ${secretsDir}/lamassu-db-password }; hostname = mkOption { @@ -155,21 +131,30 @@ in ''; }; + # Certificate options (same pattern as LND) + # TODO: When using an IP address, hostname and certificate.extraIPs are redundant. + # Consider auto-populating certificate.extraIPs from hostname if it's an IP, + # or unifying these options. For now, set both to the same IP address. certificate = { extraIPs = mkOption { type = with types; listOf str; default = []; example = [ "192.168.1.100" ]; - description = "Extra IP addresses to include in the certificate SAN."; + description = '' + Extra IP addresses to include in the certificate SAN. + ''; }; extraDomains = mkOption { type = with types; listOf str; default = []; example = [ "lamassu.example.com" ]; - description = "Extra domain names to include in the certificate SAN."; + description = '' + Extra domain names to include in the certificate SAN. + ''; }; }; + # Read-only options for certificate paths certPath = mkOption { readOnly = true; default = "${secretsDir}/lamassu-cert"; @@ -181,20 +166,41 @@ in default = "${secretsDir}/lamassu-key"; description = "Path to the TLS private key."; }; + + # NOTE: nginx is disabled for now because admin UI binds directly to port 443. + # Enabling nginx would cause a port conflict. + # Future: Add --ui-port to upstream, run admin UI on internal port (e.g., 8070), + # and use nginx as reverse proxy on 443. See docs/lamassu-future-nginx.md + # nginx = { + # enable = mkEnableOption "Nginx reverse proxy on port 443"; + # hostname = mkOption { + # type = types.nullOr types.str; + # default = null; + # description = "Hostname for nginx virtual host"; + # }; + # }; + + enableBitcoin = mkOption { + type = types.bool; + default = false; + description = "Enable Bitcoin integration (requires bitcoind)"; + }; }; config = mkIf cfg.enable { - # Secrets + # ═══════════════════════════════════════════════════════════════════════════ + # nix-bitcoin secrets integration + # ═══════════════════════════════════════════════════════════════════════════ + nix-bitcoin.secrets = { lamassu-key.user = cfg.user; lamassu-cert = { user = cfg.user; - permissions = "444"; + permissions = "444"; # World readable (it's a public cert) }; lamassu-db-password = { user = cfg.user; - group = "postgres"; - permissions = "440"; # Allow postgres group to read + group = "postgres"; # PostgreSQL needs to read this too }; }; @@ -203,42 +209,58 @@ in makePasswordSecret lamassu-db-password ''; - # PostgreSQL + # ═══════════════════════════════════════════════════════════════════════════ + + # NOTE: Nginx reverse proxy is disabled. See docs/lamassu-future-nginx.md + # for future implementation when --ui-port is added to upstream. + + # Enable PostgreSQL services.postgresql = { enable = true; package = pkgs.postgresql_15; ensureDatabases = [ cfg.database.name ]; - ensureUsers = [{ - name = cfg.database.user; - ensureDBOwnership = true; - }]; + ensureUsers = [ + { + name = cfg.database.user; + ensureDBOwnership = true; + } + ]; + # Enable password authentication for localhost connections authentication = pkgs.lib.mkOverride 10 '' + # TYPE DATABASE USER ADDRESS METHOD local all all peer host all all 127.0.0.1/32 md5 host all all ::1/128 md5 ''; }; - # User and group - users.users.${cfg.user} = { - isSystemUser = true; - group = cfg.group; - home = cfg.dataDir; - createHome = true; + # Create system users and groups + users.users = { + # Lamassu server user + ${cfg.user} = { + isSystemUser = true; + group = cfg.group; + home = cfg.dataDir; + createHome = true; + }; }; + users.groups.${cfg.group} = {}; - # Data directories + # Create data directory with proper permissions systemd.tmpfiles.rules = [ "d '${cfg.dataDir}' 0770 ${cfg.user} ${cfg.group} - -" + "d '${cfg.dataDir}/logs' 0770 ${cfg.user} ${cfg.group} - -" + "d '${cfg.dataDir}/blockchain' 0770 ${cfg.user} ${cfg.group} - -" "d '${cfg.dataDir}/ofac' 0770 ${cfg.user} ${cfg.group} - -" "d '${cfg.dataDir}/photos' 0770 ${cfg.user} ${cfg.group} - -" "d '${cfg.dataDir}/photos/idcards' 0770 ${cfg.user} ${cfg.group} - -" "d '${cfg.dataDir}/photos/frontcamera' 0770 ${cfg.user} ${cfg.group} - -" "d '${cfg.dataDir}/operator' 0770 ${cfg.user} ${cfg.group} - -" + # Source directory is created by lamassu-build service via git clone ]; - # PostgreSQL password setup + # Service to set PostgreSQL password from nix-bitcoin secrets systemd.services.lamassu-postgres-setup = { description = "Setup PostgreSQL password for lamassu-server"; wantedBy = [ "multi-user.target" ]; @@ -250,21 +272,23 @@ in User = "postgres"; }; script = '' + # Wait for user to exist, then set password from secrets for i in {1..30}; do if ${pkgs.postgresql}/bin/psql -tAc "SELECT 1 FROM pg_roles WHERE rolname='${cfg.database.user}'" | grep -q 1; then + echo "Setting password for ${cfg.database.user}..." password=$(cat ${secretsDir}/lamassu-db-password) - # Escape single quotes by doubling them (SQL standard) - escaped_password=$(printf '%s' "$password" | sed "s/'/''''/g") - ${pkgs.postgresql}/bin/psql -c "ALTER USER \"${cfg.database.user}\" WITH PASSWORD '$escaped_password';" + ${pkgs.postgresql}/bin/psql -c "ALTER USER \"${cfg.database.user}\" WITH PASSWORD '$password';" exit 0 fi + echo "Waiting for user ${cfg.database.user} to be created (attempt $i/30)..." sleep 1 done + echo "ERROR: User ${cfg.database.user} was not created after 30 seconds" exit 1 ''; }; - # Build service + # Build service - clones source and runs pnpm install/build on target systemd.services.lamassu-build = { description = "Clone and Build Lamassu Server"; wantedBy = [ "multi-user.target" ]; @@ -272,18 +296,38 @@ in wants = [ "network-online.target" ]; path = with pkgs; [ - nodejs_22 nodePackages.pnpm python3 git coreutils bash util-linux - stdenv.cc gnumake pkg-config binutils expat + nodejs_22 + nodePackages.pnpm + python3 + git + coreutils + gnused + util-linux # for setsid + # Native build tools for node-gyp (required for utf-8-validate, bufferutil, etc.) + stdenv.cc # Full C/C++ toolchain with headers + gnumake + pkg-config + binutils # ar, ranlib, etc. + # Common native dependencies for Node.js modules + libuv + openssl + # Additional dependencies for some npm packages + expat # for node-expat ]; environment = { + # Tell node-gyp where to find Python PYTHON = "${pkgs.python3}/bin/python3"; + # Ensure HOME is set for npm/pnpm cache HOME = cfg.dataDir; + # CRITICAL: pnpm fails without TTY unless CI=true is set + # See: https://github.com/pnpm/pnpm/issues/6434 CI = "true"; + # Set CC/CXX for node-gyp CC = "${pkgs.stdenv.cc}/bin/cc"; CXX = "${pkgs.stdenv.cc}/bin/c++"; - # Use content-addressable store to reduce disk usage - npm_config_cache = "${cfg.dataDir}/.npm-cache"; + # Limit concurrent scripts to avoid race conditions + npm_config_jobs = "1"; }; serviceConfig = { @@ -291,131 +335,294 @@ in RemainAfterExit = true; User = cfg.user; Group = cfg.group; + # Build can take a while, especially on first run TimeoutStartSec = "30min"; + # Don't kill child processes when main process exits KillMode = "process"; + # Send SIGTERM instead of SIGINT KillSignal = "SIGTERM"; - # Sandboxing with write access to data directory - ProtectSystem = "strict"; - ProtectHome = true; - NoNewPrivileges = true; - ReadWritePaths = [ cfg.dataDir ]; - # node-gyp needs writable /tmp for native module compilation - PrivateTmp = true; + # Completely disable sandboxing for build (npm scripts need full access) + PrivateTmp = false; + PrivateDevices = false; + ProtectSystem = false; + ProtectHome = false; + NoNewPrivileges = false; + ProtectKernelTunables = false; + ProtectKernelModules = false; + ProtectControlGroups = false; + RestrictNamespaces = false; + RestrictSUIDSGID = false; + LockPersonality = false; + # Don't restrict syscalls + SystemCallFilter = ""; + # No resource limits + TasksMax = "infinity"; + MemoryMax = "infinity"; }; script = '' set -euo pipefail - SOURCE_DIR="${cfg.dataDir}/source" - # Clone or update + SOURCE_DIR="${cfg.package}" + GIT_URL="${cfg.source.url}" + GIT_REF="${cfg.source.ref}" + + echo "==> Source: $GIT_URL (ref: $GIT_REF)" + echo "==> Target: $SOURCE_DIR" + + # Clone or update the repository if [ ! -d "$SOURCE_DIR/.git" ]; then - git clone "${cfg.source.url}" "$SOURCE_DIR" + echo "==> Cloning repository..." + git clone "$GIT_URL" "$SOURCE_DIR" cd "$SOURCE_DIR" - git checkout "${cfg.source.ref}" + git checkout "$GIT_REF" NEEDS_BUILD=1 else cd "$SOURCE_DIR" + echo "==> Fetching updates..." git fetch origin - LOCAL=$(git rev-parse HEAD) - REMOTE=$(git rev-parse "origin/${cfg.source.ref}" 2>/dev/null || git rev-parse "${cfg.source.ref}") - if [ "$LOCAL" != "$REMOTE" ]; then - git checkout "${cfg.source.ref}" - git pull origin "${cfg.source.ref}" 2>/dev/null || true + + # Check if we need to update + LOCAL_REF=$(git rev-parse HEAD) + REMOTE_REF=$(git rev-parse "origin/$GIT_REF" 2>/dev/null || git rev-parse "$GIT_REF") + + if [ "$LOCAL_REF" != "$REMOTE_REF" ]; then + echo "==> Updating to $GIT_REF..." + git checkout "$GIT_REF" + git pull origin "$GIT_REF" 2>/dev/null || true NEEDS_BUILD=1 else + echo "==> Already at latest commit: $LOCAL_REF" NEEDS_BUILD=0 fi fi - # Check build artifacts - [ ! -d "node_modules" ] || [ ! -d "packages/admin-ui/build" ] && NEEDS_BUILD=1 + # Check if build artifacts exist + if [ ! -d "node_modules" ] || [ ! -d "packages/admin-ui/build" ] || [ ! -L "packages/server/public" ]; then + echo "==> Build artifacts missing, build needed" + NEEDS_BUILD=1 + fi - [ "$NEEDS_BUILD" = "0" ] && exit 0 + if [ "$NEEDS_BUILD" = "0" ]; then + echo "==> Everything up to date, skipping build" + exit 0 + fi - # Install dependencies (without running install scripts) - pnpm install --no-frozen-lockfile --ignore-scripts + echo "==> Installing dependencies with pnpm (without scripts)..." + setsid pnpm install --no-frozen-lockfile --ignore-scripts - # Build native modules explicitly (pnpm rebuild has signal handling issues) - for module in node-expat iconv; do - path=$(find node_modules/.pnpm -name "$module" -type d -path "*/$module" 2>/dev/null | head -1) - [ -n "$path" ] && [ -f "$path/binding.gyp" ] && (cd "$path" && npx node-gyp rebuild) || true - done + echo "==> Running native module builds..." + # Run rebuild separately - this compiles native modules + setsid pnpm rebuild || echo "Warning: Some native modules failed to build, continuing anyway..." - # argon2 uses node-pre-gyp - path=$(find node_modules/.pnpm -name "argon2" -type d -path "*/argon2" 2>/dev/null | head -1) - [ -n "$path" ] && (cd "$path" && npx node-pre-gyp install --fallback-to-build) || true + echo "==> Building project..." + setsid pnpm build - # Build with setsid to isolate from signal issues - setsid --wait ./node_modules/.bin/turbo build + echo "==> Linking admin UI static files..." + cd packages/server + if [ -L public ]; then + rm public + fi + ln -s ../admin-ui/build public - # Link admin UI - ln -sfn ../admin-ui/build packages/server/public + echo "==> Build complete!" ''; }; - # Main server + # Main lamassu server service systemd.services.lamassu-server = { description = "Lamassu Bitcoin ATM Server"; wantedBy = [ "multi-user.target" ]; after = [ "network.target" "postgresql.service" "lamassu-postgres-setup.service" "lamassu-build.service" "nix-bitcoin-secrets.target" ]; wants = [ "postgresql.service" "lamassu-postgres-setup.service" "lamassu-build.service" ]; - environment = commonEnv // { + environment = { + NODE_ENV = "production"; + + # Database configuration (password read at runtime from secrets) + POSTGRES_HOST = "127.0.0.1"; + POSTGRES_PORT = "5432"; + POSTGRES_DB = cfg.database.name; + POSTGRES_USER = cfg.database.user; + + # Server configuration SERVER_PORT = toString cfg.serverPort; + LOG_LEVEL = cfg.logLevel; + HOSTNAME = cfg.hostname; + + # SSL/TLS certificates (from nix-bitcoin secrets) + CA_PATH = cfg.certPath; + CERT_PATH = cfg.certPath; + KEY_PATH = cfg.keyPath; + + # Wallet and mnemonic + MNEMONIC_PATH = "${cfg.dataDir}/lamassu-mnemonic"; + + # Data directories + OFAC_DATA_DIR = "${cfg.dataDir}/ofac"; + ID_PHOTO_CARD_DIR = "${cfg.dataDir}/photos/idcards"; + FRONT_CAMERA_DIR = "${cfg.dataDir}/photos/frontcamera"; + OPERATOR_DATA_DIR = "${cfg.dataDir}/operator"; + + # Bitcoin RPC configuration (if enabled) + BTC_NODE_LOCATION = "remote"; + BTC_WALLET_LOCATION = "remote"; + BTC_NODE_USER = "lamassu"; + BTC_NODE_RPC_HOST = "192.168.0.34"; + BTC_NODE_RPC_PORT = "8332"; + BTC_NODE_PASSWORD = "L3XF8iUrr5FNk2k6mILI"; + + # Security + SKIP_2FA = if cfg.skip2FA then "true" else "false"; }; - serviceConfig = hardeningConfig // { - WorkingDirectory = "${cfg.dataDir}/source/packages/server"; + serviceConfig = let + lamassuEnv = pkgs.writeShellScript "lamassu-env" '' + #!/bin/bash + set -euo pipefail + export PATH=${pkgs.nodejs_22}/bin:$PATH + # Read database password from nix-bitcoin secrets + DB_PASSWORD=$(cat ${secretsDir}/lamassu-db-password) + export DATABASE_URL="postgresql://${cfg.database.user}:$DB_PASSWORD@127.0.0.1:5432/${cfg.database.name}" + export POSTGRES_PASSWORD="$DB_PASSWORD" + export NODE_PATH=${cfg.package}/node_modules:${cfg.package}/packages/server/node_modules + cd ${cfg.package} + exec "$@" + ''; + in defaultHardening // { + WorkingDirectory = "${cfg.package}/packages/server"; ExecStartPre = [ - "${pkgs.bash}/bin/bash -c 'test -f ${cfg.dataDir}/lamassu-mnemonic || echo \"abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about\" > ${cfg.dataDir}/lamassu-mnemonic'" + # Generate BIP39 mnemonic if it doesn't exist + "${pkgs.bash}/bin/bash -c 'if [[ ! -f ${cfg.dataDir}/lamassu-mnemonic ]]; then echo \"abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about\" > ${cfg.dataDir}/lamassu-mnemonic && chmod 600 ${cfg.dataDir}/lamassu-mnemonic; fi'" + # Run database migration "${lamassuEnv} ${pkgs.nodejs_22}/bin/node packages/server/bin/lamassu-migrate" ]; ExecStart = "${lamassuEnv} ${pkgs.nodejs_22}/bin/node packages/server/bin/lamassu-server --port ${toString cfg.serverPort} --logLevel ${cfg.logLevel}"; + + # Node.js specific overrides + MemoryDenyWriteExecute = false; + + # Allow read/write access + ReadWritePaths = [ cfg.dataDir cfg.package "/tmp" ]; + RestrictAddressFamilies = [ "AF_UNIX" "AF_INET" "AF_INET6" ]; + + # Service identity + User = cfg.user; + Group = cfg.group; + Restart = "on-failure"; + RestartSec = "10s"; }; preStart = '' + mkdir -p ${cfg.dataDir}/logs + + # Wait for PostgreSQL using peer authentication timeout=30 while ! ${pkgs.postgresql}/bin/psql -h /run/postgresql -U ${cfg.database.user} -d ${cfg.database.name} -c '\q' 2>/dev/null; do - [ $timeout -le 0 ] && exit 1 + if [ $timeout -le 0 ]; then + echo "Timeout waiting for PostgreSQL" + exit 1 + fi + echo "Waiting for PostgreSQL..." sleep 1 ((timeout--)) done + echo "PostgreSQL is ready" ''; }; - # Admin server + # Admin server service systemd.services.lamassu-admin-server = { description = "Lamassu Admin Server"; wantedBy = [ "multi-user.target" ]; after = [ "network.target" "lamassu-server.service" "lamassu-build.service" ]; wants = [ "lamassu-server.service" "lamassu-build.service" ]; - environment = commonEnv; + environment = { + NODE_ENV = "production"; + LOG_LEVEL = cfg.logLevel; + HOSTNAME = cfg.hostname; + CA_PATH = cfg.certPath; + CERT_PATH = cfg.certPath; + KEY_PATH = cfg.keyPath; + # Database configuration (password read at runtime from secrets) + POSTGRES_HOST = "127.0.0.1"; + POSTGRES_PORT = "5432"; + POSTGRES_DB = cfg.database.name; + POSTGRES_USER = cfg.database.user; + MNEMONIC_PATH = "${cfg.dataDir}/lamassu-mnemonic"; + SKIP_2FA = if cfg.skip2FA then "true" else "false"; + # Data directories + OFAC_DATA_DIR = "${cfg.dataDir}/ofac"; + ID_PHOTO_CARD_DIR = "${cfg.dataDir}/photos/idcards"; + FRONT_CAMERA_DIR = "${cfg.dataDir}/photos/frontcamera"; + OPERATOR_DATA_DIR = "${cfg.dataDir}/operator"; + }; - serviceConfig = hardeningConfig // { - WorkingDirectory = "${cfg.dataDir}/source/packages/server"; - ExecStart = "${lamassuEnv} ${pkgs.nodejs_22}/bin/node packages/server/bin/lamassu-admin-server --logLevel ${cfg.logLevel}"; + serviceConfig = let + lamassuAdminEnv = pkgs.writeShellScript "lamassu-admin-env" '' + #!/bin/bash + set -euo pipefail + export PATH=${pkgs.nodejs_22}/bin:$PATH + # Read database password from nix-bitcoin secrets + DB_PASSWORD=$(cat ${secretsDir}/lamassu-db-password) + export DATABASE_URL="postgresql://${cfg.database.user}:$DB_PASSWORD@127.0.0.1:5432/${cfg.database.name}" + export POSTGRES_PASSWORD="$DB_PASSWORD" + export NODE_PATH=${cfg.package}/node_modules:${cfg.package}/packages/admin-server/node_modules + cd ${cfg.package} + exec "$@" + ''; + in defaultHardening // { + WorkingDirectory = "${cfg.package}/packages/server"; + ExecStart = "${lamassuAdminEnv} ${pkgs.nodejs_22}/bin/node packages/server/bin/lamassu-admin-server --logLevel ${cfg.logLevel}"; + MemoryDenyWriteExecute = false; + ReadWritePaths = [ cfg.dataDir cfg.package ]; + User = cfg.user; + Group = cfg.group; + Restart = "on-failure"; + RestartSec = "10s"; + + # Allow binding to privileged port 443 (production mode) AmbientCapabilities = [ "CAP_NET_BIND_SERVICE" ]; CapabilityBoundingSet = [ "CAP_NET_BIND_SERVICE" ]; }; }; - # Firewall + # Open firewall ports + # Port 3000 (configurable): machine API access (required for pairing and operation) + # Port 443: admin UI (production mode, hardcoded in upstream) networking.firewall.allowedTCPPorts = [ cfg.serverPort 443 ]; - # Helper tools + # Add useful packages environment.systemPackages = with pkgs; [ nodejs_22 nodePackages.pnpm postgresql (writeShellScriptBin "lamassu-register-user" '' + # Read database password from nix-bitcoin secrets DB_PASSWORD=$(cat ${secretsDir}/lamassu-db-password) + export NODE_PATH="${cfg.package}/node_modules:${cfg.package}/packages/server/node_modules" export DATABASE_URL="postgresql://${cfg.database.user}:$DB_PASSWORD@127.0.0.1:5432/${cfg.database.name}" - export POSTGRES_HOST="127.0.0.1" POSTGRES_PORT="5432" - export POSTGRES_DB="${cfg.database.name}" POSTGRES_USER="${cfg.database.user}" POSTGRES_PASSWORD="$DB_PASSWORD" - export HOSTNAME="${cfg.hostname}" SKIP_2FA="${if cfg.skip2FA then "true" else "false"}" - export NODE_ENV="${cfg.mode}" - sudo -E -u ${cfg.user} ${pkgs.nodejs_22}/bin/node ${cfg.dataDir}/source/packages/server/bin/lamassu-register "$@" + export HOSTNAME="${cfg.hostname}" + export POSTGRES_HOST="127.0.0.1" + export POSTGRES_PORT="5432" + export POSTGRES_DB="${cfg.database.name}" + export POSTGRES_USER="${cfg.database.user}" + export POSTGRES_PASSWORD="$DB_PASSWORD" + export SKIP_2FA="${if cfg.skip2FA then "true" else "false"}" + + sudo -E -u ${cfg.user} bash -c "cd ${cfg.package}/packages/server && ${pkgs.nodejs_22}/bin/node bin/lamassu-register \"\$@\"" -- "$@" + '') + (writeShellScriptBin "lamassu-status" '' + echo "=== Lamassu Server Status ===" + systemctl status lamassu-server lamassu-admin-server + echo "" + echo "=== Database Status ===" + sudo -u ${cfg.database.user} ${pkgs.postgresql}/bin/psql -d ${cfg.database.name} -c "SELECT version();" + echo "" + echo "=== Network Access ===" + echo "Server API: https://localhost:${toString cfg.serverPort}" + echo "Admin UI: https://localhost:443" '') ]; };