feat: price a back-dated period at the day it was due

Until now every payout converted at whatever the rate was when it settled,
so a period paid late was silently mispriced. Contracts now carry a pricing
mode, and every payout records the rate it used plus where that rate came
from — a figure in the ledger can be explained months later instead of
merely trusted.

Three modes, per contract and overridable per payout:

- `payday` (default) converts at what BTC was worth on the payday itself,
  via the historical lookup.
- `current` converts at today's rate — correct when the obligation reads
  "we owe EUR 800 whenever it settles".
- `manual` converts at a rate the operator states (100000 EUR/BTC), for a
  figure that was agreed rather than looked up.

For a payday that is today or ahead, all three collapse to the same thing
and none of them touches the network: LNbits' own live pricing is the
freshest source available, so `resolve_price` returns the fiat amount
unconverted and lets create_invoice do its job. History is consulted only
where it can actually change the answer.

Where payroll does convert, it must hand create_invoice a sat amount —
create_invoice always prices fiat itself and cannot be told a rate. That
moves the single conversion point into payroll, which is why the rate and
its source are recorded on the payout. In `current` mode the rate is read
back off the invoice LNbits priced (extra["btc_rate"]) rather than
recomputed, so the row records the number actually applied.

An unavailable rate raises PricingError and fails the period. Deliberately
no fallback to today's rate: a rate that moved 30% since the payday would
pay 30% off and hide it, which is the class of error nobody finds until an
audit. The existing retry-then-pause machinery already handles a failed
period, and the ledger row names the date and currency that could not be
priced. Manual mode missing its rate is caught at contract-creation time
instead, rather than surfacing as a failed payout weeks later.

m003 defaults preserve behaviour for anything in flight — every live payday
is today or ahead, where the modes agree. Verified by applying m003 to a
copy of the running instance's database: the existing contract and its paid
payout both survive and read back correctly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018jy52j9GRZ6XKa1Zt21LLj
This commit is contained in:
Padreug 2026-08-31 21:42:00 +02:00
commit 18a17cd693
7 changed files with 345 additions and 12 deletions

View file

@ -88,3 +88,27 @@ async def m002_payouts(db):
"CREATE INDEX payroll.idx_payouts_employee_wallet " "CREATE INDEX payroll.idx_payouts_employee_wallet "
"ON payouts (employee_wallet);" "ON payouts (employee_wallet);"
) )
async def m003_pricing_mode(db):
"""Let a back-dated period be priced at the day it was due.
Until now every payout converted at whatever the rate was when it
settled, which silently misprices a period paid late. The contract now
carries how to convert, and each payout records the rate it actually
used plus where that rate came from — so a figure in the ledger can be
explained months later instead of merely trusted.
Defaults preserve existing behaviour for anything already in flight:
every live payday is today or ahead, and for those all three modes agree
on using LNbits' own live pricing.
"""
await db.execute(
"ALTER TABLE payroll.contracts "
"ADD COLUMN pricing_mode TEXT NOT NULL DEFAULT 'payday';"
)
await db.execute("ALTER TABLE payroll.contracts ADD COLUMN manual_rate REAL;")
await db.execute("ALTER TABLE payroll.payouts ADD COLUMN rate REAL;")
await db.execute(
"ALTER TABLE payroll.payouts ADD COLUMN rate_source TEXT NOT NULL DEFAULT '';"
)

View file

@ -55,6 +55,25 @@ class Frequency(str, Enum):
yearly = "yearly" yearly = "yearly"
class PricingMode(str, Enum):
"""How a period's fiat amount becomes a sat amount.
Only ever matters for a payday in the past — for a payday that is today
or ahead, all three agree and LNbits' own live pricing is used.
"""
# Convert at what BTC was worth on the payday itself. The default: a
# back-dated period is priced as of when it was due, not when somebody
# got round to paying it.
payday = "payday"
# Convert at today's rate. Correct when the obligation is understood as
# "we owe them EUR 800, whenever it settles".
current = "current"
# Convert at a rate the operator states outright (e.g. 100000 EUR/BTC),
# for a figure that was agreed rather than looked up.
manual = "manual"
class ContractStatus(str, Enum): class ContractStatus(str, Enum):
active = "active" # eligible for payout on every scheduler tick active = "active" # eligible for payout on every scheduler tick
paused = "paused" # keeps its schedule position, pays nothing paused = "paused" # keeps its schedule position, pays nothing
@ -101,6 +120,19 @@ class CreateContract(BaseModel):
# months of back-pay in one tick. # months of back-pay in one tick.
backfill: bool = False backfill: bool = False
# How to price a back-dated period. Irrelevant for a sat-denominated
# contract, and irrelevant for any payday that is not in the past.
pricing_mode: PricingMode = PricingMode.payday
# Units of `currency` per whole BTC. Required by, and only read under,
# PricingMode.manual.
manual_rate: float | None = None
@validator("manual_rate")
def _positive_rate(cls, v: float | None) -> float | None:
if v is not None and v <= 0:
raise ValueError("manual_rate must be positive")
return v
@validator("start_date") @validator("start_date")
def _valid_date(cls, v: str) -> str: def _valid_date(cls, v: str) -> str:
datetime.strptime(v, "%Y-%m-%d") # raises for anything malformed datetime.strptime(v, "%Y-%m-%d") # raises for anything malformed
@ -159,6 +191,8 @@ class UpdateContract(BaseModel):
total_periods: int | None = None total_periods: int | None = None
label: str | None = None label: str | None = None
memo: str | None = None memo: str | None = None
pricing_mode: PricingMode | None = None
manual_rate: float | None = None
@validator("amount") @validator("amount")
def _positive_amount(cls, v: float | None) -> float | None: def _positive_amount(cls, v: float | None) -> float | None:
@ -239,6 +273,14 @@ class Payout(BaseModel):
payment_hash: str | None = None payment_hash: str | None = None
detail: str = "" detail: str = ""
# Units of `currency` per whole BTC actually used for this period, and
# which of the pricing modes produced it. For a live conversion the rate
# is read back off the invoice LNbits priced rather than recomputed, so
# the row records what happened and not an approximation of it. Null on
# a sat contract, and on a failure that never got as far as pricing.
rate: float | None = None
rate_source: str = "" # "current" | "payday" | "manual" | "" (sat / none)
created_at: datetime = Field(default_factory=_now) created_at: datetime = Field(default_factory=_now)
@property @property

View file

@ -33,7 +33,9 @@ from .models import (
Frequency, Frequency,
Payout, Payout,
PayoutStatus, PayoutStatus,
PricingMode,
) )
from .rates import historical_btc_rate, sats_for
# Frequencies that are an exact number of days: no calendar involved, so no # Frequencies that are an exact number of days: no calendar involved, so no
# clamping is possible or needed. # clamping is possible or needed.
@ -187,6 +189,8 @@ class PeriodOutcome:
detail: str = "" detail: str = ""
amount_msat: int | None = None amount_msat: int | None = None
payment_hash: str | None = None payment_hash: str | None = None
rate: float | None = None
rate_source: str = ""
@property @property
def consumed(self) -> bool: def consumed(self) -> bool:
@ -214,12 +218,86 @@ def _lock_for(contract_id: str) -> asyncio.Lock:
return lock return lock
def _today() -> date:
return datetime.now(timezone.utc).date()
def payout_memo(contract: Contract, payday: date) -> str: def payout_memo(contract: Contract, payday: date) -> str:
label = contract.memo or contract.label or "Payroll" label = contract.memo or contract.label or "Payroll"
return f"{label} — {payday.isoformat()}" return f"{label} — {payday.isoformat()}"
async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutcome: class PricingError(ValueError):
"""A period's sat amount could not be established.
Deliberately fatal for that period. The alternative — falling back to
today's rate when the payday's rate is unavailable — pays a materially
different amount than intended and hides it, which is exactly the class
of error nobody finds until an audit.
"""
@dataclass
class Price:
"""What to hand `create_invoice`, plus the provenance to record.
When `currency` is the contract's own, LNbits does the conversion and
`rate` is filled in afterwards from the invoice it priced — read back
rather than recomputed, so the ledger records what actually happened.
"""
amount: float
currency: str
rate: float | None
source: str
async def resolve_price(
contract: Contract,
payday: date,
today: date,
mode: PricingMode | None = None,
manual_rate: float | None = None,
) -> Price:
"""Decide what a period is worth, and say where the number came from.
`mode`/`manual_rate` override the contract's own settings for a single
call, which is how an operator prices one off-cycle payout differently
without editing the contract.
"""
if contract.currency.lower() == "sat":
return Price(contract.amount, "sat", None, "")
mode = mode or contract.pricing_mode
manual_rate = manual_rate or contract.manual_rate
# A payday that is today or ahead needs no history: every mode agrees,
# and LNbits' own live pricing is the freshest source available.
if payday >= today or mode == PricingMode.current:
return Price(contract.amount, contract.currency, None, "current")
if mode == PricingMode.manual:
if not manual_rate:
raise PricingError("manual pricing selected but no rate given")
return Price(
sats_for(contract.amount, manual_rate), "sat", manual_rate, "manual"
)
rate = await historical_btc_rate(payday, contract.currency)
if rate is None:
raise PricingError(
f"no historical {contract.currency} rate available for {payday}"
)
return Price(sats_for(contract.amount, rate), "sat", rate, "payday")
async def pay_period(
contract: Contract,
index: int,
payday: date,
mode: PricingMode | None = None,
manual_rate: float | None = None,
) -> PeriodOutcome:
"""Move one period's money from the source wallet to the employee wallet. """Move one period's money from the source wallet to the employee wallet.
An internal invoice on the destination wallet, paid from the source An internal invoice on the destination wallet, paid from the source
@ -240,11 +318,23 @@ async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutc
"payday": payday.isoformat(), "payday": payday.isoformat(),
} }
try:
price = await resolve_price(contract, payday, _today(), mode, manual_rate)
except (PricingError, ValueError) as exc:
return PeriodOutcome(
index=index,
payday=payday,
status=PayoutStatus.failed,
detail=str(exc),
)
if price.source:
extra["rate_source"] = price.source
try: try:
invoice = await create_invoice( invoice = await create_invoice(
wallet_id=contract.employee_wallet, wallet_id=contract.employee_wallet,
amount=contract.amount, amount=price.amount,
currency=contract.currency, currency=price.currency,
memo=memo, memo=memo,
internal=True, internal=True,
extra=extra, extra=extra,
@ -262,6 +352,13 @@ async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutc
# invoice.amount is msat and is now the canonical figure for this period. # invoice.amount is msat and is now the canonical figure for this period.
amount_msat = invoice.amount amount_msat = invoice.amount
# For a live conversion LNbits stashes the rate it used on the payment;
# read it back rather than recomputing, so the ledger records the number
# that was actually applied.
rate = price.rate
if rate is None and price.source == "current":
rate = (invoice.extra or {}).get("btc_rate")
source = await get_wallet(contract.source_wallet) source = await get_wallet(contract.source_wallet)
if not source: if not source:
return PeriodOutcome( return PeriodOutcome(
@ -270,6 +367,8 @@ async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutc
status=PayoutStatus.failed, status=PayoutStatus.failed,
detail="source wallet not found", detail="source wallet not found",
amount_msat=amount_msat, amount_msat=amount_msat,
rate=rate,
rate_source=price.source,
) )
if source.balance_msat < amount_msat: if source.balance_msat < amount_msat:
return PeriodOutcome( return PeriodOutcome(
@ -282,6 +381,8 @@ async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutc
f"{amount_msat // 1000} sat required" f"{amount_msat // 1000} sat required"
), ),
amount_msat=amount_msat, amount_msat=amount_msat,
rate=rate,
rate_source=price.source,
) )
try: try:
@ -300,6 +401,8 @@ async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutc
detail=f"payment failed: {exc}", detail=f"payment failed: {exc}",
amount_msat=amount_msat, amount_msat=amount_msat,
payment_hash=invoice.payment_hash, payment_hash=invoice.payment_hash,
rate=rate,
rate_source=price.source,
) )
return PeriodOutcome( return PeriodOutcome(
@ -308,6 +411,8 @@ async def pay_period(contract: Contract, index: int, payday: date) -> PeriodOutc
status=PayoutStatus.paid, status=PayoutStatus.paid,
amount_msat=amount_msat, amount_msat=amount_msat,
payment_hash=invoice.payment_hash, payment_hash=invoice.payment_hash,
rate=rate,
rate_source=price.source,
) )
@ -324,7 +429,7 @@ async def run_due_periods(
retried on the next tick, because a failed period does not advance the retried on the next tick, because a failed period does not advance the
contract's position. contract's position.
""" """
today = today or datetime.now(timezone.utc).date() today = today or _today()
async with _lock_for(contract.id): async with _lock_for(contract.id):
# Re-read under the lock: a manual run may have moved the position # Re-read under the lock: a manual run may have moved the position
@ -386,6 +491,8 @@ async def _record(contract: Contract, outcome: PeriodOutcome) -> Payout:
source_wallet=contract.source_wallet, source_wallet=contract.source_wallet,
payment_hash=outcome.payment_hash, payment_hash=outcome.payment_hash,
detail=outcome.detail, detail=outcome.detail,
rate=outcome.rate,
rate_source=outcome.rate_source,
) )
) )
@ -522,7 +629,7 @@ def resume(
contract.status = ContractStatus.active contract.status = ContractStatus.active
if not catch_up: if not catch_up:
today = today or datetime.now(timezone.utc).date() today = today or _today()
skipped_to = fast_forward_index(contract, today) skipped_to = fast_forward_index(contract, today)
if skipped_to != contract.periods_done: if skipped_to != contract.periods_done:
logger.info( logger.info(
@ -551,7 +658,11 @@ def cancel(contract: Contract) -> Contract:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
async def pay_now(contract: Contract) -> Payout: async def pay_now(
contract: Contract,
mode: PricingMode | None = None,
manual_rate: float | None = None,
) -> Payout:
"""Settle the contract's next period immediately, whatever the calendar """Settle the contract's next period immediately, whatever the calendar
says. says.
@ -565,6 +676,10 @@ async def pay_now(contract: Contract) -> Payout:
because `_settle`'s back-dated skip exists to stop a *new* contract because `_settle`'s back-dated skip exists to stop a *new* contract
firing surprise back-pay. An operator explicitly asking to pay a period firing surprise back-pay. An operator explicitly asking to pay a period
is not a surprise. is not a surprise.
`mode`/`manual_rate` price this one payout differently without editing
the contract — the case an operator hits when entering a payment that
happened weeks ago at a rate they already know.
""" """
async with _lock_for(contract.id): async with _lock_for(contract.id):
fresh = await crud.get_contract(contract.id) fresh = await crud.get_contract(contract.id)
@ -582,9 +697,9 @@ async def pay_now(contract: Contract) -> Payout:
index = contract.periods_done index = contract.periods_done
payday = occurrence_on(parse_start_date(contract), contract.frequency, index) payday = occurrence_on(parse_start_date(contract), contract.frequency, index)
today = datetime.now(timezone.utc).date() today = _today()
outcome = await pay_period(contract, index, payday) outcome = await pay_period(contract, index, payday, mode, manual_rate)
if payday > today and outcome.status == PayoutStatus.paid: if payday > today and outcome.status == PayoutStatus.paid:
outcome.detail = f"paid early on {today.isoformat()} (due {payday})" outcome.detail = f"paid early on {today.isoformat()} (due {payday})"

View file

@ -7,7 +7,7 @@ keeping `services` split into a pure half and an effectful half.
from datetime import datetime, timezone from datetime import datetime, timezone
from ..models import Contract, ContractStatus, Frequency from ..models import Contract, ContractStatus, Frequency, PricingMode
def make_contract( def make_contract(
@ -22,6 +22,8 @@ def make_contract(
currency: str = "sat", currency: str = "sat",
backfill: bool = False, backfill: bool = False,
created_at: str = "2026-01-01", created_at: str = "2026-01-01",
pricing_mode: PricingMode = PricingMode.payday,
manual_rate: float | None = None,
) -> Contract: ) -> Contract:
return Contract( return Contract(
id=contract_id, id=contract_id,
@ -37,6 +39,8 @@ def make_contract(
periods_done=periods_done, periods_done=periods_done,
status=status, status=status,
backfill=backfill, backfill=backfill,
pricing_mode=pricing_mode,
manual_rate=manual_rate,
created_at=datetime.strptime(created_at, "%Y-%m-%d").replace( created_at=datetime.strptime(created_at, "%Y-%m-%d").replace(
tzinfo=timezone.utc tzinfo=timezone.utc
), ),

View file

@ -57,7 +57,7 @@ def payroll_stub(monkeypatch):
async def fake_count_period_failures(_contract_id, _period_index): async def fake_count_period_failures(_contract_id, _period_index):
return stub.prior_failures return stub.prior_failures
async def fake_pay_period(contract, index, payday): async def fake_pay_period(contract, index, payday, mode=None, manual_rate=None):
stub.attempted.append(index) stub.attempted.append(index)
paid = stub.outcome_status == PayoutStatus.paid paid = stub.outcome_status == PayoutStatus.paid
return PeriodOutcome( return PeriodOutcome(

131
tests/test_pricing.py Normal file
View file

@ -0,0 +1,131 @@
"""Pricing a period.
`resolve_price` is where a back-dated payout stops being worth today's
money. These pin down which mode wins when, and that an unavailable rate
refuses rather than substitutes.
"""
import asyncio
from datetime import date
import pytest
from .. import services
from ..models import PricingMode
from ..services import PricingError, resolve_price
from .conftest import make_contract
TODAY = date(2026, 8, 31)
PAST = date(2026, 5, 15)
PAST_RATE = 69743.26
@pytest.fixture
def historical(monkeypatch):
"""Stub the lookup and record what it was asked for."""
asked = []
async def fake_rate(day, currency):
asked.append((day, currency))
return PAST_RATE if currency == "EUR" else None
monkeypatch.setattr(services, "historical_btc_rate", fake_rate)
return asked
def eur(**kw):
return make_contract(currency="EUR", amount=800, **kw)
def price(contract, payday=PAST, today=TODAY, **kw):
return asyncio.run(resolve_price(contract, payday, today, **kw))
# --- when history is irrelevant --------------------------------------------
def test_a_payday_today_uses_live_pricing(historical):
"""No mode needs history for a payday that has not passed — LNbits' own
conversion is the freshest source there is."""
p = price(eur(), payday=TODAY)
assert (p.currency, p.source) == ("EUR", "current")
assert p.amount == 800 # handed to create_invoice as fiat, unconverted
assert historical == []
def test_a_future_payday_uses_live_pricing(historical):
p = price(eur(), payday=date(2026, 12, 1))
assert p.source == "current"
assert historical == []
def test_a_sat_contract_never_prices_at_all(historical):
p = price(make_contract(currency="sat", amount=1000))
assert (p.amount, p.currency, p.rate, p.source) == (1000, "sat", None, "")
assert historical == []
# --- the three modes -------------------------------------------------------
def test_payday_mode_converts_at_the_paydays_rate(historical):
p = price(eur(pricing_mode=PricingMode.payday))
assert p.source == "payday"
assert p.currency == "sat"
assert p.rate == PAST_RATE
assert p.amount == round(800 / PAST_RATE * 1e8)
assert historical == [(PAST, "EUR")]
def test_current_mode_ignores_the_past_payday(historical):
"""The employee is owed EUR 800 and gets EUR 800 of bitcoin today."""
p = price(eur(pricing_mode=PricingMode.current))
assert (p.currency, p.source) == ("EUR", "current")
assert historical == []
def test_manual_mode_uses_the_stated_rate(historical):
p = price(eur(pricing_mode=PricingMode.manual, manual_rate=100_000))
assert p.source == "manual"
assert p.rate == 100_000
assert p.amount == 800_000 # 800 EUR at 100k EUR/BTC == 0.008 BTC
assert historical == []
def test_manual_mode_without_a_rate_refuses(historical):
with pytest.raises(PricingError):
price(eur(pricing_mode=PricingMode.manual))
# --- per-call override -----------------------------------------------------
def test_an_override_beats_the_contract(historical):
"""Pricing one off-cycle payout differently must not require editing the
contract."""
contract = eur(pricing_mode=PricingMode.payday)
p = price(contract, mode=PricingMode.manual, manual_rate=50_000)
assert (p.source, p.rate) == ("manual", 50_000)
assert historical == []
def test_an_override_rate_supplies_a_contract_that_has_none(historical):
p = price(eur(), mode=PricingMode.manual, manual_rate=50_000)
assert p.amount == round(800 / 50_000 * 1e8)
# --- the refusal that matters ----------------------------------------------
def test_an_unavailable_rate_refuses_rather_than_substituting(historical):
"""The whole point: no silent fallback to today's rate. A rate that
moved 30% since the payday would otherwise pay 30% off, discovered only
on review."""
with pytest.raises(PricingError):
price(make_contract(currency="JPY", amount=800))
def test_the_refusal_names_the_date_and_currency(historical):
with pytest.raises(PricingError) as exc:
price(make_contract(currency="JPY", amount=800))
assert "JPY" in str(exc.value) and "2026-05-15" in str(exc.value)

View file

@ -30,6 +30,7 @@ from .models import (
DirectoryUser, DirectoryUser,
Payout, Payout,
PayoutStatus, PayoutStatus,
PricingMode,
SchedulePreview, SchedulePreview,
SchedulePreviewRequest, SchedulePreviewRequest,
UpdateContract, UpdateContract,
@ -95,6 +96,14 @@ async def _validate_terms(data: CreateContract) -> str:
HTTPStatus.BAD_REQUEST, f"Unsupported currency '{data.currency}'." HTTPStatus.BAD_REQUEST, f"Unsupported currency '{data.currency}'."
) )
# Catch this now rather than as a failed payout weeks later, when the
# first back-dated period tries to price itself and finds no rate.
if data.pricing_mode == PricingMode.manual and not data.manual_rate:
raise HTTPException(
HTTPStatus.BAD_REQUEST,
"Manual pricing needs a rate (units of the currency per BTC).",
)
return employee.display_name return employee.display_name
@ -301,7 +310,11 @@ async def api_contract_schedule(contract_id: str, count: int = 12) -> SchedulePr
@payroll_api_router.post("/api/v1/contracts/{contract_id}/pay-now") @payroll_api_router.post("/api/v1/contracts/{contract_id}/pay-now")
async def api_pay_now(contract_id: str) -> Payout: async def api_pay_now(
contract_id: str,
pricing_mode: PricingMode | None = None,
manual_rate: float | None = None,
) -> Payout:
"""Settle the next period immediately, whatever the calendar says. """Settle the next period immediately, whatever the calendar says.
Covers both "run it now" (do not wait for the tick) and "pay it early", Covers both "run it now" (do not wait for the tick) and "pay it early",
@ -312,12 +325,16 @@ async def api_pay_now(contract_id: str) -> Payout:
Recorded in the ledger like any other payout, with the early-payment Recorded in the ledger like any other payout, with the early-payment
noted in its detail. Returns the ledger row, including a failed one: noted in its detail. Returns the ledger row, including a failed one:
the caller wants to know *why* a manual payout did not land. the caller wants to know *why* a manual payout did not land.
`pricing_mode`/`manual_rate` price this one payout differently without
editing the contract — for entering a payment that happened weeks ago at
a rate the operator already knows. They apply to this call only.
""" """
contract = await crud.get_contract(contract_id) contract = await crud.get_contract(contract_id)
if not contract: if not contract:
raise HTTPException(HTTPStatus.NOT_FOUND, "Contract not found.") raise HTTPException(HTTPStatus.NOT_FOUND, "Contract not found.")
try: try:
return await services.pay_now(contract) return await services.pay_now(contract, pricing_mode, manual_rate)
except services.LifecycleError as exc: except services.LifecycleError as exc:
raise HTTPException(HTTPStatus.CONFLICT, str(exc)) from exc raise HTTPException(HTTPStatus.CONFLICT, str(exc)) from exc