claude-forgejo-sandbox/docs/reforge.md
Padreug df0fd9a9ba feat: extract reforge engine into a standalone consumable flake
The forgejo-sandbox / reforge harness, lifted out of the machine config
into a host-agnostic, generic engine anyone can consume with Nix.

Two layers:
- engine (this repo) — nixosModules.reforge stands up the sandbox forge,
  provisions role accounts + tokens, enforces branch protection, and puts
  the reforge-* CLI + forgejo-mcp on PATH. Carries no project specifics.
- run config — per-project manifest/charter/agenda/issues an adopter fills
  in; scaffold one with the `reforge` flake template.

Portability fixes vs the in-config version:
- forgejo-mcp resolved from $REFORGE_MCP_BIN or PATH, never a named host
  (kills the nixosConfigurations.omni hardcode).
- all instance data + paths parameterized via REFORGE_* env, baked into the
  reforge-scripts wrappers from module options (configDir, agentsDir,
  refsDir, org, port, tokenOwner, ...).
- option namespace neutral (reforge.* not omni.packs.*); settings policies
  carry no absolute /etc/nixos paths.
- role briefs + orchestrator playbook genericized: all project specifics
  point at the charter; refs corpus optional.

Validated: nix flake check (eval) + builds of forgejo-mcp, reforge-scripts,
and a module-eval check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 22:51:47 +02:00

9.7 KiB
Raw Permalink Blame History

Reforge — stack-rebuild simulations

A reforge is a controlled simulation in which a role-isolated agent team rebuilds a software stack from pinned clean bases to byte-for-byte parity with a declared target — the known working state — coordinating entirely through a local forge (issues, branches, PRs, enforced independent review), exactly as a real development team would.

The infrastructure is the forgejo sandbox (docs/architecture.md); this doc is the concept and the lifecycle.

What it actually is

Several familiar practices, fused:

  • A disaster-recovery drill for the codebase. Proves the stack can be re-derived from public upstream bases plus recorded intent — bus-factor insurance for software whose value lives partly in accumulated, under-documented decisions.
  • A review retrofit. Every line of the stack re-passes through a security gate and a charter-alignment gate that didn't exist when it was first written. Code that "works in production" earns its bill of health instead of inheriting it.
  • An insight harvest. The issues, review comments, and rejections produced during the rebuild are findings about the real stack, free of production pressure. They are first-class artifacts, archived with every run.
  • Adjacent to clean-room reimplementation, but inverted: the target is declared and diffed against, rather than hidden. Fidelity is measured, not hoped for.

A run's input parameters

A simulation is fully specified by five things — everything else is produced by the run, not fed into it. All five live in your run config dir (reforge.configDir), so a run's parameters are reproducible by checkout:

  1. The repo list — which pieces of the stack are in scope (manifest.txt rows).
  2. The charter — the standard changes are judged against (charter.md, seeded as the charter repo).
  3. Each repo's starting condition — the pinned base it is seeded at (base_ref; for original software: empty).
  4. Each repo's ending condition — the target state it must reach byte-for-byte (target_url + target_ref).
  5. The agenda — what this particular run must address (agenda.md, seeded as charter/AGENDA.md, filed as issues from issues.tsv at kickoff).

Disclosure policy (staged visibility)

The end conditions are a functioning proof of concept — the run exists to find its bugs, refactor opportunities, and distance from production before release. To keep the findings honest, what the role sessions may see is staged:

  • Phase A — guided rebuild, end-state blind. Roles work from intent, not answers: the charter + agenda (in-forge) and whatever intent docs the charter points to. The target repos are off-limits; issues and design decisions produced in this phase reflect what the team would build from the recorded intent.
  • Phase B — end-state visible, convergence and evolution. Targets are opened to the roles (as source-stripped local mirrors — see below). Phase B has two objectives: (1) converge each repo to its target (compare script as the gate), re-derived through gated PRs so every line re-passes the security + charter gate; and (2) evolve toward the vision — carry the Phase-A findings forward as tracked divergences (approved issue + gated PR + manifest-target bump; never silent drift) that move the stack toward the charter. Fixes the target does not contain are parked open (do-not-merge, kept visible as remediation), never closed.
  • Comparing ambition levels. To see the result of each scope of evolution, build them as separate runs from ONE shared parity baseline: reach parity → archive the baseline → drive one scope to completion → archive it → restore the baseline and drive the next. The archives are the comparison corpus: identical inputs, divergent ambition.
  • Both phases: targets are never pushed into the sandbox. Only the compare script touches them mechanically, and only the operator runs it (roles and the orchestrator are barred from the real remotes). Phase B visibility is provided by reforge-fetch-targets, which clones each target into a source-stripped local mirror (~/reforge-targets/<repo>, git metadata removed → no remote, no push path) that roles read as the end-state.

The human decides when Phase A has produced enough independent signal to open Phase B — that switch is a log-worthy moment of the run.

Vocabulary

Term Meaning
simulation / run One full pass: reset → seed → rebuild → compare → archive
base The pinned upstream state a fork is seeded from (base_ref in the manifest)
target The known working state (real repo + branch) the run must reproduce byte-for-byte
manifest manifest.txt — repo set, bases, targets
acceptance reforge-compare reporting IDENTICAL for every targeted repo
archive Cold tarball of the full forge state (repos + issues + PRs + reviews) per run

The contract (invariants of a real run)

  1. Bases are pinned. No auto refs in the manifest — two seedings must produce the same starting point.
  2. Targets are declared but never pushed into the sandbox. Only the compare script touches them; content parity is judged on trees, not history — the simulation re-derives the tree through its own PR history.
  3. All change flows through PRs. Direct push to main is blocked everywhere; only security-lead's approval unlocks a merge; the reviewer holds quality + charter alignment.
  4. Acceptance is mechanical. The run is done when the compare script exits 0 — byte-for-byte, no "close enough". The targets are a PoC, so parity includes reproducing its bugs: findings become issues, not divergence. A truly justified divergence is an approved issue plus a manifest-target update, never a silent drift.
  5. Findings are first-class deliverables. Issues traceable to agenda items, reviews on every PR, and a GAMEPLAN.md (PR'd into the charter repo) stating what stands between the stack and production-ready.
  6. Runs are never destroyed, only closed. reset archives before it wipes; archives are the corpus future runs (and the real stack) learn from.

Lifecycle

0. pin      edit manifest.txt in your config dir        (bases + targets)
1. reset    reforge-reset reset <run-name>
            (archives the previous simulation, wipes to zero, reprovisions
             users/tokens/org — then get the new admin UI password from
             /var/lib/forgejo-sandbox/admin-password)
2. seed     reforge-seed
3. kickoff  reforge-kickoff
            (files issues.tsv as [item-id]-tagged issues; roles start in
            Phase A, end-state blind)
4. rebuild  role sessions do the work — one terminal per role:
            reforge-role <role>
            issues to plan, branches + PRs to change, reviews to gate;
            human opens Phase B when Phase A's signal is banked
5. compare  reforge-compare
            iterate 4 ⇄ 5 until every targeted repo is IDENTICAL
6. harvest  GAMEPLAN.md PR'd into charter; issues reviewed + triaged against
            the real stack; **refine these instruction sources** (this doc,
            the agent briefs, the agenda, the scripts) with what the run
            taught — the simulation is meant to improve every iteration
7. close    reforge-reset backup <run-name>
            (or go straight to the next run's reset, which archives too)

Archives: /var/lib/forgejo-sandbox-archive/<name>-<stamp>.tar.gz; restore any of them with reforge-reset restore <tarball>.

Autonomous mode (an agent as orchestrator)

Steps 35 (kickoff → rebuild → compare) can be driven by an agent itself instead of a human rotating tabs. The orchestrator does the operator job — schedule, route, judge convergence, checkpoint — and launches fresh headless role sessions for all real work. Isolation is preserved because it coordinates only through the forge, by issue/PR number, never by content; it never reviews or implements.

reforge-orchestrator
# then: "Drive Phase A to completion, stopping at the A->B boundary."

Playbook: agents/orchestrator.md (seeded as the session's CLAUDE.md). Policy: scripts/settings/orchestrator-settings.json.

Design properties that make this safe to let run:

  • The gate is structural, not enforced by the orchestrator. Admin can't push through branch protection (verified), the orchestrator's policy denies curl/git push/reset, and the forge MCP exposes no branch-protection tool — so the worst a runaway orchestrator does is file bad issues/PRs, never merge unreviewed code to a target.
  • Merges are gated, not bypassed. It merges via merge_pull_request only a PR already carrying a security-lead APPROVED review.
  • Sequential role turns — one actor at a time, no forge races.
  • Auditable — it keeps a "Run log — orchestrator" issue in charter, one line per scheduling/judgment call.
  • Human-stop at Phase A → B — the one consequential, hard-to-reverse judgment is handed back, not taken autonomously.

Caveats, stated honestly:

  • Nested headless turns cost real tokens and compound; give it a bounded instruction ("Phase A only") and watch the run log.
  • Model non-determinism compounds across many turns: the inputs (pins, charter, agenda) are reproducible, but the trajectory is not. Treat each autonomous run as one sample — archive it, and you can compare two runs from identical inputs.
  • For a clean "how does it do from scratch" run, restore a post-kickoff/pre-work archive first (reset → seed → kickoff → backup run-N-kickoff-clean), then launch the orchestrator against that.