aiolabs/bitspire
9
0
Fork 0
Code Issues 52 Pull requests 2 Projects Releases Packages Wiki Activity Actions

fix(deploy): nightly auto-upgrade failed on both ATMs, for different reasons #101

Merged
padreug merged 2 commits from fix/autoupgrade-known-hosts-and-wg into dev 2026-09-22 18:28:38 +00:00
Conversation 0 Commits 2 Files changed 1 +26
padreug commented 2026-09-22 18:25:21 +00:00
Owner
Copy link

Closes #98

batm3 never got past fetching. system.autoUpgrade runs as root, root had no known_hosts entry for git.atitlan.io, and every nightly run died at "Host key verification failed" before authentication. It sat on its 2026-08-06 install generation for six weeks. The host key is now declared in the config, so a freshly flashed ATM updates from first boot with no manual step. Verified against the key sintra's root already trusts.

sintra failed at the last step, over something unrelated to updating. Its WireGuard tunnel was never provisioned, so wireguard-wg0 fails on every activation, and one failed unit makes switch-to-configuration exit 4, marking the whole run failed even though the generation applied. The unit is now skipped when there is no key. Guarded on wg0 still being declared so the live image, which mkForce's the interfaces away, doesn't inherit a unit with no ExecStart.

Both system closures build. Note that batm3 needs one manual step before its updater works: its root key is not authorized on the repo, so the run will then fail at authentication. Key in the issue.

Closes #98 batm3 never got past fetching. system.autoUpgrade runs as root, root had no known_hosts entry for git.atitlan.io, and every nightly run died at "Host key verification failed" before authentication. It sat on its 2026-08-06 install generation for six weeks. The host key is now declared in the config, so a freshly flashed ATM updates from first boot with no manual step. Verified against the key sintra's root already trusts. sintra failed at the last step, over something unrelated to updating. Its WireGuard tunnel was never provisioned, so wireguard-wg0 fails on every activation, and one failed unit makes switch-to-configuration exit 4, marking the whole run failed even though the generation applied. The unit is now skipped when there is no key. Guarded on wg0 still being declared so the live image, which mkForce's the interfaces away, doesn't inherit a unit with no ExecStart. Both system closures build. Note that batm3 needs one manual step before its updater works: its root key is not authorized on the repo, so the run will then fail at authentication. Key in the issue.
system.autoUpgrade fetches the flake over ssh as root. A machine whose
root has never connected by hand has no known_hosts entry, so the run
dies at 'Host key verification failed' before it even reaches
authentication. batm3 did exactly that, silently, from its 2026-08-06
install until 09-22: six weeks on its install generation while a unit
nobody was watching reported failure every night. sintra only ever
worked because a human had ssh'd as root once and accepted the key.

Declaring the key means a freshly flashed ATM updates from first boot
with no manual step. Verified against the key sintra's root already
trusts.

Refs #98

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
wg0.key is written per machine after flashing. Until it is, the unit's
`wg set … private-key` exits 1 with 'fopen: No such file or directory',
and one failed unit makes switch-to-configuration exit 4 — which marks
the whole nightly system.autoUpgrade run as failed even though the new
generation applied. sintra has reported a broken updater on that basis
alone; its tunnel was never provisioned and wg0 has never existed.

Skip the unit when there is no key rather than failing activation over
an interface that was never set up. A provisioned machine is unaffected.
Guarded on wg0 still being declared so the live image, which mkForce's
the interfaces away, doesn't inherit a unit with no ExecStart.

Refs #98

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
padreug deleted branch fix/autoupgrade-known-hosts-and-wg 2026-09-22 18:28:38 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
aiolabs/bitspire!101
No description provided.