53 lines
4.4 KiB
Markdown
53 lines
4.4 KiB
Markdown
# Burrow Forge Runbook
|
|
|
|
This directory contains the Burrow forge host definition and the Hetzner bootstrap shape for `burrow-forge`.
|
|
|
|
Mail hosting is intentionally not part of this NixOS host in the current plan. Burrow's first mail path is Forward Email with Burrow-owned custom S3 backups; see [`docs/FORWARDEMAIL.md`](../docs/FORWARDEMAIL.md).
|
|
|
|
## Files
|
|
|
|
- `hosts/burrow-forge/default.nix`: host entrypoint
|
|
- `modules/burrow-forge.nix`: Forgejo, Caddy, PostgreSQL, and admin bootstrap module
|
|
- `modules/burrow-forge-runner.nix`: Forgejo Actions runner and agent identity bootstrap
|
|
- `modules/burrow-forgejo-nsc.nix`: Namespace-backed ephemeral Forgejo runner services
|
|
- `hetzner-cloud-config.yaml`: desired Hetzner host shape
|
|
- `keys/contact_at_burrow_net.pub`: initial operator SSH public key
|
|
- `keys/agent_at_burrow_net.pub`: automation SSH public key
|
|
- `../Scripts/hetzner-forge.sh`: Hetzner inventory and replace workflow
|
|
- `../Scripts/nsc-build-and-upload-image.sh`: temporary Namespace builder -> raw image -> Hetzner snapshot
|
|
- `../Scripts/bootstrap-forge-intake.sh`: legacy intake bootstrap helper; current forge runtime secrets should live in `../secrets/forgejo/*.age`
|
|
- `../Scripts/check-forge-host.sh`: verify Forgejo, Caddy, the local runner, and optional NSC services after boot
|
|
- `../Scripts/cloudflare-upsert-a-record.sh`: upsert DNS-only Cloudflare `A` records for Burrow host cutovers
|
|
- `../Scripts/forge-deploy.sh`: remote `nixos-rebuild` entrypoint for the forge host
|
|
- `../Scripts/provision-forgejo-nsc.sh`: render Burrow Namespace dispatcher/autoscaler bootstrap inputs and ensure the default Forgejo scope exists
|
|
- `../secrets/forgejo/*.age`: authoritative encrypted forge admin password, agent SSH key, and Namespace runtime configs for the forge host
|
|
|
|
## Intended Flow
|
|
|
|
1. Build and upload the raw NixOS image with `Scripts/hetzner-forge.sh build-image` or `Scripts/nsc-build-and-upload-image.sh`.
|
|
2. Recreate `burrow-forge` from the latest labeled snapshot with `Scripts/hetzner-forge.sh recreate-from-image --yes`.
|
|
3. Encrypt the Forgejo admin password and agent SSH key into `secrets/forgejo/{admin-password,agent-ssh-key}.age`.
|
|
4. Let `burrow-forgejo-bootstrap.service` create or rotate the initial Forgejo admin account from the agenix secret path.
|
|
5. Let `burrow-forgejo-runner-bootstrap.service` register the self-hosted Forgejo runner and seed Git identity as `agent <agent@burrow.net>`.
|
|
6. Run `Scripts/provision-forgejo-nsc.sh` locally to refresh `secrets/forgejo/{nsc-token,nsc-dispatcher-config,nsc-autoscaler-config}.age`, then deploy with `Scripts/forge-deploy.sh` so agenix updates the live forgejo-nsc runtime paths.
|
|
7. Use `Scripts/cloudflare-upsert-a-record.sh` to point `git.burrow.net`, `burrow.net`, and `nsc-autoscaler.burrow.net` at the host with Cloudflare proxying disabled for ACME.
|
|
8. Use `Scripts/forge-deploy.sh --allow-dirty` for subsequent remote `nixos-rebuild` runs from the live workspace.
|
|
9. Configure Forward Email custom S3 backups for `burrow.net` and `burrow.rs` out-of-band with `Tools/forwardemail-custom-s3.sh`.
|
|
|
|
## Current Constraints
|
|
|
|
- `burrow-forge` is live on NixOS in `hel1` at `89.167.47.21`, and `Scripts/check-forge-host.sh --expect-nsc` passes locally against that host.
|
|
- Public Burrow forge cutover completed on March 15, 2026:
|
|
- `burrow.net`, `git.burrow.net`, and `nsc-autoscaler.burrow.net` now publish public `A` records to `89.167.47.21`
|
|
- HTTP redirects to HTTPS on all three names
|
|
- `https://burrow.net` returns the root forge landing response
|
|
- `https://git.burrow.net` returns the live Forgejo front door
|
|
- `https://nsc-autoscaler.burrow.net` terminates TLS on Caddy and returns the expected application-level `404` for `/`
|
|
- The Cloudflare token now lives in `secrets/cloudflare/api-token.age`; the current token is account-scoped: `POST /accounts/<account>/tokens/verify` succeeds, while `POST /user/tokens/verify` returns `Invalid API Token`.
|
|
- `burrow.rs` still resolves publicly to a Vercel `DEPLOYMENT_NOT_FOUND` response.
|
|
- Both domains publish Forward Email MX/TXT records.
|
|
- Forward Email custom S3 is live on both domains against the Hetzner `burrow` bucket and the public regional endpoint `https://hel1.your-objectstorage.com`.
|
|
- The current Hetzner account contains both:
|
|
- the older Ubuntu bootstrap server in `hil`
|
|
- the live `burrow-forge` NixOS server in `hel1`
|
|
- The remaining forge work is follow-on product/integration work, not host bring-up, mail backup wiring, or public DNS cutover.
|