Wire Forge-native release infrastructure
Some checks failed
Cache: Publish Nix / Publish Nix Cache (push) Waiting to run
Build Rust / Cargo Test (push) Successful in 4m14s
Build Site / Next.js Build (push) Failing after 6s
Infra: OpenTofu / OpenTofu (grafana) (push) Successful in 5s
Lint Governance / BEP Metadata (push) Successful in 0s
Build: Android / Android Rust Core Stub (push) Failing after 23s

This commit is contained in:
Conrad Kramer 2026-06-07 05:51:12 -07:00
parent 97c569fb35
commit 002bd382e9
199 changed files with 14268 additions and 185 deletions

88
docs/NIX_CACHE.md Normal file
View file

@ -0,0 +1,88 @@
# Burrow Nix Cache
`nix.burrow.net` is the Burrow Nix binary cache surface for forge jobs and
workers.
The host layout is:
- Garage stores objects on the forge host's Garage data volume.
- Attic serves the Nix binary cache API at `nix.burrow.net`.
- Attic stores NAR chunks in the Garage `attic` bucket through the local S3 API.
- Workers consume `https://nix.burrow.net/burrow` as a substituter once the
cache public key is configured.
Garage is not a multi-backend object sync manager. It can replicate across
Garage nodes, but it does not write the same object to GCS and another provider
as independent storage backends. Multi-cloud backup should be a separate mirror
or export job from Garage buckets to GCS and later to additional providers.
## Host Bootstrap
`services.burrow.garage` creates a host-local environment file at
`/var/lib/burrow/garage/env`. The file is not stored in Git or the Nix store.
It contains:
- Garage RPC secret
- Garage access keys for Attic, release artifacts, package repositories, and
read-only backups
- Attic JWT signing secret
- AWS-compatible variables used by Attic to talk to Garage
`burrow-garage-bootstrap.service` creates the initial single-node Garage layout,
imports the generated keys, and creates the `attic`, `burrow-releases`, and
`burrow-packages` buckets. It grants the backup key read-only access to those
three buckets so CI can mirror them without receiving owner/write access to
Garage.
`services.burrow.nixCache` enables Attic, points it at Garage, exposes it through
Caddy, creates a public `burrow` cache, and writes two host-local tokens:
- `/var/lib/burrow/nix-cache/admin-token`: bootstrap/admin recovery token
- `/var/lib/burrow/nix-cache/ci-push-token`: scoped pull/push token for the
`burrow` cache
## Worker Configuration
`Scripts/ci/ensure-nix.sh` uses the Burrow cache only when
`BURROW_NIX_CACHE_PUBLIC_KEY` is present:
```sh
BURROW_NIX_CACHE_URL=https://nix.burrow.net/burrow
BURROW_NIX_CACHE_PUBLIC_KEY=<cache public key from attic cache info>
```
Without the public key, CI falls back to `cache.nixos.org` only. This avoids
configuring an unsigned or untrusted substituter.
After deployment, get the public key from the host:
```sh
attic cache info local:burrow
```
Then set `BURROW_NIX_CACHE_PUBLIC_KEY` in Forgejo variables.
Seal the scoped push token from `/var/lib/burrow/nix-cache/ci-push-token` into
Forgejo/OpenBao as `BURROW_NIX_CACHE_PUSH_TOKEN`. The
`Cache: Publish Nix` workflow runs `Scripts/ci/publish-nix-cache.sh`, builds the
selected flake outputs, and pushes their closures to Attic. It skips cleanly
when the push token is absent unless `BURROW_NIX_CACHE_REQUIRED=true`.
## Backups
The Garage `attic` bucket is backed up to the private GCS bucket
`burrow-net-nix-cache`. Release artifacts and package repositories are backed up
to their public GCS buckets. The scheduled `Backup: Garage Storage` workflow
runs `Scripts/ci/backup-garage-to-gcs.sh` with:
```sh
BURROW_GARAGE_ENDPOINT=https://objects.burrow.net
AWS_ACCESS_KEY_ID=<GARAGE_BACKUP_ACCESS_KEY_ID from the host env>
AWS_SECRET_ACCESS_KEY=<GARAGE_BACKUP_SECRET_ACCESS_KEY from the host env>
BURROW_NIX_CACHE_GCS_BUCKET=burrow-net-nix-cache
```
The workflow authenticates to Google through Authentik-backed WIF and uses
`gcloud storage rsync`. It does not use rclone or Google service-account JSON
keys. Destination deletes are disabled by default; set
`BURROW_GARAGE_BACKUP_DELETE=true` only for an intentional prune run.