Add Docs
This commit is contained in:
parent
1d2eb05c36
commit
862fe0bfad
4 changed files with 2729 additions and 0 deletions
386
docs/SATS-EQUIVALENT-METADATA.md
Normal file
386
docs/SATS-EQUIVALENT-METADATA.md
Normal file
|
|
@ -0,0 +1,386 @@
|
|||
# SATS-Equivalent Metadata Field
|
||||
|
||||
**Date**: 2025-01-12
|
||||
**Status**: Current Architecture
|
||||
**Location**: Beancount posting metadata
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The `sats-equivalent` metadata field is Castle's solution for **dual-currency tracking** in a fiat-denominated ledger. It preserves Bitcoin (satoshi) amounts alongside fiat currency amounts without violating accounting principles or creating multi-currency complexity in position balances.
|
||||
|
||||
### Quick Summary
|
||||
|
||||
- **Purpose**: Track Bitcoin/Lightning amounts in a EUR-denominated ledger
|
||||
- **Location**: Beancount posting metadata (not position amounts)
|
||||
- **Format**: String containing absolute satoshi amount (e.g., `"337096"`)
|
||||
- **Primary Use**: Calculate user balances in satoshis (Castle's primary currency)
|
||||
- **Key Principle**: Satoshis are for reference; EUR is the actual transaction currency
|
||||
|
||||
---
|
||||
|
||||
## The Problem: Dual-Currency Tracking
|
||||
|
||||
Castle needs to track both:
|
||||
1. **Fiat amounts** (EUR, USD) - The actual transaction currency
|
||||
2. **Bitcoin amounts** (satoshis) - The Lightning Network settlement currency
|
||||
|
||||
### Why Not Just Use SATS as Position Amounts?
|
||||
|
||||
**Accounting Reality**: When a user pays €36.93 cash for groceries, the transaction is denominated in EUR, not Bitcoin. Recording it as Bitcoin would:
|
||||
- ❌ Misrepresent the actual transaction
|
||||
- ❌ Create exchange rate volatility issues
|
||||
- ❌ Complicate traditional accounting reconciliation
|
||||
- ❌ Make fiat-based reporting difficult
|
||||
|
||||
**Castle's Philosophy**: Record transactions in their **actual currency**, with Bitcoin as supplementary data.
|
||||
|
||||
---
|
||||
|
||||
## Architecture: EUR-Primary Format
|
||||
|
||||
### Current Ledger Format
|
||||
|
||||
```beancount
|
||||
2025-11-10 * "Groceries (36.93 EUR)" #expense-entry
|
||||
Expenses:Food:Supplies 36.93 EUR
|
||||
sats-equivalent: "39669"
|
||||
reference: "cash-payment-abc123"
|
||||
Liabilities:Payable:User-5987ae95 -36.93 EUR
|
||||
sats-equivalent: "39669"
|
||||
```
|
||||
|
||||
**Key Components:**
|
||||
- **Position Amount**: `36.93 EUR` - The actual transaction amount
|
||||
- **Metadata**: `sats-equivalent: "39669"` - The Bitcoin equivalent at time of transaction
|
||||
- **Sign**: The sign (debit/credit) is on the EUR amount; sats-equivalent is always absolute value
|
||||
|
||||
### How It's Created
|
||||
|
||||
In `views_api.py:839`:
|
||||
|
||||
```python
|
||||
# If fiat currency is provided, use EUR-based format
|
||||
if fiat_currency and fiat_amount:
|
||||
# EUR-based posting (current architecture)
|
||||
posting_metadata["sats-equivalent"] = str(abs(line.amount))
|
||||
|
||||
# Apply the sign from line.amount to fiat_amount
|
||||
signed_fiat_amount = fiat_amount if line.amount >= 0 else -fiat_amount
|
||||
|
||||
posting = {
|
||||
"account": account.name,
|
||||
"amount": f"{signed_fiat_amount:.2f} {fiat_currency}",
|
||||
"meta": posting_metadata if posting_metadata else None
|
||||
}
|
||||
```
|
||||
|
||||
**Critical Details:**
|
||||
- `line.amount` is always in satoshis internally
|
||||
- The sign (debit/credit) transfers to the fiat amount
|
||||
- `sats-equivalent` stores the **absolute value** of the satoshi amount
|
||||
- Sign interpretation depends on account type (Asset/Liability/etc.)
|
||||
|
||||
---
|
||||
|
||||
## Usage: Balance Calculation
|
||||
|
||||
### Primary Use Case: User Balances
|
||||
|
||||
Castle's core function is tracking **who owes whom** in satoshis. The `sats-equivalent` metadata enables this.
|
||||
|
||||
**Flow** (`fava_client.py:220-248`):
|
||||
|
||||
```python
|
||||
# Parse posting amount (EUR/USD)
|
||||
fiat_match = re.match(r'^(-?[\d.]+)\s+([A-Z]{3})$', amount_str)
|
||||
if fiat_match and fiat_match.group(2) in ('EUR', 'USD', 'GBP'):
|
||||
fiat_amount = Decimal(fiat_match.group(1))
|
||||
fiat_currency = fiat_match.group(2)
|
||||
|
||||
# Track fiat balance
|
||||
fiat_balances[fiat_currency] += fiat_amount
|
||||
|
||||
# Extract SATS equivalent from metadata
|
||||
posting_meta = posting.get("meta", {})
|
||||
sats_equiv = posting_meta.get("sats-equivalent")
|
||||
if sats_equiv:
|
||||
# Apply the sign from fiat_amount to sats_equiv
|
||||
sats_amount = int(sats_equiv) if fiat_amount > 0 else -int(sats_equiv)
|
||||
total_sats += sats_amount
|
||||
```
|
||||
|
||||
**Sign Interpretation:**
|
||||
- EUR amount is `36.93` (positive/debit) → sats is `+39669`
|
||||
- EUR amount is `-36.93` (negative/credit) → sats is `-39669`
|
||||
|
||||
### Secondary Use: Journal Entry Display
|
||||
|
||||
When displaying transactions to users (`views_api.py:747-751`):
|
||||
|
||||
```python
|
||||
# Extract sats equivalent from metadata
|
||||
posting_meta = first_posting.get("meta", {})
|
||||
sats_equiv = posting_meta.get("sats-equivalent")
|
||||
if sats_equiv:
|
||||
amount_sats = abs(int(sats_equiv))
|
||||
```
|
||||
|
||||
This allows the UI to show both EUR and SATS amounts for each transaction.
|
||||
|
||||
---
|
||||
|
||||
## Why Metadata Instead of Positions?
|
||||
|
||||
### The BQL Limitation
|
||||
|
||||
Beancount Query Language (BQL) **cannot access metadata**. This means:
|
||||
|
||||
```sql
|
||||
-- ✅ This works (queries position amounts):
|
||||
SELECT account, sum(position) WHERE account ~ 'User-5987ae95'
|
||||
-- Returns: EUR positions (not useful for satoshi balances)
|
||||
|
||||
-- ❌ This is NOT possible:
|
||||
SELECT account, sum(meta["sats-equivalent"]) WHERE account ~ 'User-5987ae95'
|
||||
-- Error: BQL cannot access metadata
|
||||
```
|
||||
|
||||
### Why Castle Accepts This Trade-off
|
||||
|
||||
**Performance Analysis** (see `docs/BQL-BALANCE-QUERIES.md`):
|
||||
1. **Caching solves the bottleneck**: 60-80% performance improvement from caching account/permission lookups
|
||||
2. **Iteration is necessary anyway**: Even with BQL, we'd need to iterate postings to access metadata
|
||||
3. **Manual aggregation is fast**: The actual summation is not the bottleneck
|
||||
4. **Database queries are the bottleneck**: Solved by Phase 1 caching, not BQL
|
||||
|
||||
**Architectural Correctness > Query Performance**:
|
||||
- ✅ Transactions recorded in their actual currency
|
||||
- ✅ No artificial multi-currency positions
|
||||
- ✅ Clean accounting reconciliation
|
||||
- ✅ Exchange rate changes don't affect historical records
|
||||
|
||||
---
|
||||
|
||||
## Alternative Considered: Price Notation
|
||||
|
||||
### Price Notation Format (Not Implemented)
|
||||
|
||||
```beancount
|
||||
2025-11-10 * "Groceries"
|
||||
Expenses:Food -360.00 EUR @@ 337096 SATS
|
||||
Liabilities:Payable:User-abc 360.00 EUR @@ 337096 SATS
|
||||
```
|
||||
|
||||
**Pros:**
|
||||
- ✅ BQL can query prices (enables BQL aggregation)
|
||||
- ✅ Standard Beancount syntax
|
||||
- ✅ SATS trackable via price database
|
||||
|
||||
**Cons:**
|
||||
- ❌ Semantically incorrect: `@@` means "total price paid", not "equivalent value"
|
||||
- ❌ Implies currency conversion happened (it didn't)
|
||||
- ❌ Confuses future readers about transaction nature
|
||||
- ❌ Complicates Beancount's price database
|
||||
|
||||
**Decision**: Metadata is more semantically correct for "reference value" than price notation.
|
||||
|
||||
See `docs/BQL-PRICE-NOTATION-SOLUTION.md` for full analysis.
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Example
|
||||
|
||||
### User Adds Expense
|
||||
|
||||
**User Action**: "I paid €36.93 cash for groceries"
|
||||
|
||||
**Castle's Internal Representation**:
|
||||
```python
|
||||
# User provides or Castle calculates:
|
||||
fiat_amount = Decimal("36.93") # EUR
|
||||
fiat_currency = "EUR"
|
||||
amount_sats = 39669 # Calculated from exchange rate
|
||||
|
||||
# Create journal entry line:
|
||||
line = CreateEntryLine(
|
||||
account_id=expense_account.id,
|
||||
amount=amount_sats, # Internal: always satoshis
|
||||
metadata={
|
||||
"fiat_currency": "EUR",
|
||||
"fiat_amount": "36.93"
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
**Beancount Entry Created** (`views_api.py:835-849`):
|
||||
```beancount
|
||||
2025-11-10 * "Groceries (36.93 EUR)" #expense-entry
|
||||
Expenses:Food:Supplies 36.93 EUR
|
||||
sats-equivalent: "39669"
|
||||
Liabilities:Payable:User-5987ae95 -36.93 EUR
|
||||
sats-equivalent: "39669"
|
||||
```
|
||||
|
||||
**Balance Calculation** (`fava_client.py:get_user_balance`):
|
||||
```python
|
||||
# Iterate all postings for user accounts
|
||||
# For each posting:
|
||||
# - Parse EUR amount: -36.93 EUR (credit to liability)
|
||||
# - Extract sats-equivalent: "39669"
|
||||
# - Apply sign: -36.93 is negative → sats = -39669
|
||||
# - Accumulate: user_balance_sats += -39669
|
||||
|
||||
# Result: negative balance = Castle owes user
|
||||
```
|
||||
|
||||
**User Balance Response**:
|
||||
```json
|
||||
{
|
||||
"user_id": "5987ae95",
|
||||
"balance": -39669, // Castle owes user 39,669 sats
|
||||
"fiat_balances": {
|
||||
"EUR": "-36.93" // Castle owes user €36.93
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Where It's Set
|
||||
|
||||
**Primary Location**: `views_api.py:835-849` (Creating journal entries)
|
||||
|
||||
All EUR-based postings get `sats-equivalent` metadata:
|
||||
- Expense entries (user adds liability)
|
||||
- Receivable entries (admin records what user owes)
|
||||
- Revenue entries (direct income)
|
||||
- Payment entries (settling balances)
|
||||
|
||||
### Where It's Read
|
||||
|
||||
**Primary Location**: `fava_client.py:239-247` (Balance calculation)
|
||||
|
||||
Used in:
|
||||
1. `get_user_balance()` - Calculate individual user balance
|
||||
2. `get_all_user_balances()` - Calculate all user balances
|
||||
3. `get_journal_entries()` - Display transaction amounts
|
||||
|
||||
### Data Type and Format
|
||||
|
||||
- **Type**: String (Beancount metadata values must be strings or numbers)
|
||||
- **Format**: Absolute value, no sign, no decimal point
|
||||
- **Examples**:
|
||||
- ✅ `"39669"` (correct)
|
||||
- ✅ `"1000000"` (1M sats)
|
||||
- ❌ `"-39669"` (incorrect: sign goes on EUR amount)
|
||||
- ❌ `"396.69"` (incorrect: satoshis are integers)
|
||||
|
||||
---
|
||||
|
||||
## Key Principles
|
||||
|
||||
### 1. Record in Transaction Currency
|
||||
|
||||
```beancount
|
||||
# ✅ CORRECT: User paid EUR, record in EUR
|
||||
Expenses:Food 36.93 EUR
|
||||
sats-equivalent: "39669"
|
||||
|
||||
# ❌ WRONG: Recording Bitcoin when user paid cash
|
||||
Expenses:Food 39669 SATS {36.93 EUR}
|
||||
```
|
||||
|
||||
### 2. Preserve Historical Values
|
||||
|
||||
The `sats-equivalent` is the **exact satoshi amount at transaction time**. It does NOT change when exchange rates change.
|
||||
|
||||
**Example:**
|
||||
- 2025-11-10: User pays €36.93 → 39,669 sats (rate: 1074.19 sats/EUR)
|
||||
- 2025-11-15: Exchange rate changes to 1100 sats/EUR
|
||||
- **Metadata stays**: `sats-equivalent: "39669"` ✅
|
||||
- **If we used current rate**: Would become 40,623 sats ❌
|
||||
|
||||
### 3. Separate Fiat and Sats Balances
|
||||
|
||||
Castle tracks TWO independent balances:
|
||||
- **Satoshi balance**: Sum of `sats-equivalent` metadata (primary)
|
||||
- **Fiat balances**: Sum of EUR/USD position amounts (secondary)
|
||||
|
||||
These are calculated independently and don't cross-convert.
|
||||
|
||||
### 4. Absolute Values in Metadata
|
||||
|
||||
The sign (debit/credit) lives on the position amount, NOT the metadata.
|
||||
|
||||
```beancount
|
||||
# Debit (expense increases):
|
||||
Expenses:Food 36.93 EUR # Positive
|
||||
sats-equivalent: "39669" # Absolute value
|
||||
|
||||
# Credit (liability increases):
|
||||
Liabilities:Payable -36.93 EUR # Negative
|
||||
sats-equivalent: "39669" # Same absolute value
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Path
|
||||
|
||||
### Future: If We Change to SATS-Primary Format
|
||||
|
||||
**Hypothetical future format:**
|
||||
```beancount
|
||||
; SATS as position, EUR as cost:
|
||||
2025-11-10 * "Groceries"
|
||||
Expenses:Food 39669 SATS {36.93 EUR}
|
||||
Liabilities:Payable:User-abc -39669 SATS {36.93 EUR}
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- ✅ BQL can query SATS directly
|
||||
- ✅ No metadata parsing needed
|
||||
- ✅ Standard Beancount cost accounting
|
||||
|
||||
**Migration Script** (conceptual):
|
||||
```python
|
||||
# Read all postings with sats-equivalent metadata
|
||||
# For each posting:
|
||||
# - Extract sats from metadata
|
||||
# - Extract EUR from position
|
||||
# - Rewrite as: "<sats> SATS {<eur> EUR}"
|
||||
```
|
||||
|
||||
**Decision**: Not implementing now because:
|
||||
1. Current architecture is semantically correct
|
||||
2. Performance is acceptable with caching
|
||||
3. Migration would break existing tooling
|
||||
4. EUR-primary aligns with accounting reality
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- `docs/BQL-BALANCE-QUERIES.md` - Why BQL can't query metadata and performance analysis
|
||||
- `docs/BQL-PRICE-NOTATION-SOLUTION.md` - Alternative using price notation (not implemented)
|
||||
- `beancount_format.py` - Functions that create entries with sats-equivalent metadata
|
||||
- `fava_client.py:get_user_balance()` - How metadata is parsed for balance calculation
|
||||
|
||||
---
|
||||
|
||||
## Technical Summary
|
||||
|
||||
**Field**: `sats-equivalent`
|
||||
**Type**: Metadata (string)
|
||||
**Location**: Beancount posting metadata
|
||||
**Format**: Absolute satoshi amount as string (e.g., `"39669"`)
|
||||
**Purpose**: Track Bitcoin equivalent of fiat transactions
|
||||
**Primary Use**: Calculate user satoshi balances
|
||||
**Sign Handling**: Inherits from position amount (EUR/USD)
|
||||
**Queryable via BQL**: ❌ No (BQL cannot access metadata)
|
||||
**Performance**: ✅ Acceptable with caching (60-80% improvement)
|
||||
**Architectural Status**: ✅ Current production format
|
||||
**Future Migration**: Possible to SATS-primary if needed
|
||||
Loading…
Add table
Add a link
Reference in a new issue