Wire Forge-native release infrastructure
Some checks failed
Cache: Publish Nix / Publish Nix Cache (push) Waiting to run
Build Rust / Cargo Test (push) Successful in 4m14s
Build Site / Next.js Build (push) Failing after 6s
Infra: OpenTofu / OpenTofu (grafana) (push) Successful in 5s
Lint Governance / BEP Metadata (push) Successful in 0s
Build: Android / Android Rust Core Stub (push) Failing after 23s
Some checks failed
Cache: Publish Nix / Publish Nix Cache (push) Waiting to run
Build Rust / Cargo Test (push) Successful in 4m14s
Build Site / Next.js Build (push) Failing after 6s
Infra: OpenTofu / OpenTofu (grafana) (push) Successful in 5s
Lint Governance / BEP Metadata (push) Successful in 0s
Build: Android / Android Rust Core Stub (push) Failing after 23s
This commit is contained in:
parent
97c569fb35
commit
002bd382e9
199 changed files with 14268 additions and 185 deletions
88
docs/NIX_CACHE.md
Normal file
88
docs/NIX_CACHE.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
# Burrow Nix Cache
|
||||
|
||||
`nix.burrow.net` is the Burrow Nix binary cache surface for forge jobs and
|
||||
workers.
|
||||
|
||||
The host layout is:
|
||||
|
||||
- Garage stores objects on the forge host's Garage data volume.
|
||||
- Attic serves the Nix binary cache API at `nix.burrow.net`.
|
||||
- Attic stores NAR chunks in the Garage `attic` bucket through the local S3 API.
|
||||
- Workers consume `https://nix.burrow.net/burrow` as a substituter once the
|
||||
cache public key is configured.
|
||||
|
||||
Garage is not a multi-backend object sync manager. It can replicate across
|
||||
Garage nodes, but it does not write the same object to GCS and another provider
|
||||
as independent storage backends. Multi-cloud backup should be a separate mirror
|
||||
or export job from Garage buckets to GCS and later to additional providers.
|
||||
|
||||
## Host Bootstrap
|
||||
|
||||
`services.burrow.garage` creates a host-local environment file at
|
||||
`/var/lib/burrow/garage/env`. The file is not stored in Git or the Nix store.
|
||||
It contains:
|
||||
|
||||
- Garage RPC secret
|
||||
- Garage access keys for Attic, release artifacts, package repositories, and
|
||||
read-only backups
|
||||
- Attic JWT signing secret
|
||||
- AWS-compatible variables used by Attic to talk to Garage
|
||||
|
||||
`burrow-garage-bootstrap.service` creates the initial single-node Garage layout,
|
||||
imports the generated keys, and creates the `attic`, `burrow-releases`, and
|
||||
`burrow-packages` buckets. It grants the backup key read-only access to those
|
||||
three buckets so CI can mirror them without receiving owner/write access to
|
||||
Garage.
|
||||
|
||||
`services.burrow.nixCache` enables Attic, points it at Garage, exposes it through
|
||||
Caddy, creates a public `burrow` cache, and writes two host-local tokens:
|
||||
|
||||
- `/var/lib/burrow/nix-cache/admin-token`: bootstrap/admin recovery token
|
||||
- `/var/lib/burrow/nix-cache/ci-push-token`: scoped pull/push token for the
|
||||
`burrow` cache
|
||||
|
||||
## Worker Configuration
|
||||
|
||||
`Scripts/ci/ensure-nix.sh` uses the Burrow cache only when
|
||||
`BURROW_NIX_CACHE_PUBLIC_KEY` is present:
|
||||
|
||||
```sh
|
||||
BURROW_NIX_CACHE_URL=https://nix.burrow.net/burrow
|
||||
BURROW_NIX_CACHE_PUBLIC_KEY=<cache public key from attic cache info>
|
||||
```
|
||||
|
||||
Without the public key, CI falls back to `cache.nixos.org` only. This avoids
|
||||
configuring an unsigned or untrusted substituter.
|
||||
|
||||
After deployment, get the public key from the host:
|
||||
|
||||
```sh
|
||||
attic cache info local:burrow
|
||||
```
|
||||
|
||||
Then set `BURROW_NIX_CACHE_PUBLIC_KEY` in Forgejo variables.
|
||||
|
||||
Seal the scoped push token from `/var/lib/burrow/nix-cache/ci-push-token` into
|
||||
Forgejo/OpenBao as `BURROW_NIX_CACHE_PUSH_TOKEN`. The
|
||||
`Cache: Publish Nix` workflow runs `Scripts/ci/publish-nix-cache.sh`, builds the
|
||||
selected flake outputs, and pushes their closures to Attic. It skips cleanly
|
||||
when the push token is absent unless `BURROW_NIX_CACHE_REQUIRED=true`.
|
||||
|
||||
## Backups
|
||||
|
||||
The Garage `attic` bucket is backed up to the private GCS bucket
|
||||
`burrow-net-nix-cache`. Release artifacts and package repositories are backed up
|
||||
to their public GCS buckets. The scheduled `Backup: Garage Storage` workflow
|
||||
runs `Scripts/ci/backup-garage-to-gcs.sh` with:
|
||||
|
||||
```sh
|
||||
BURROW_GARAGE_ENDPOINT=https://objects.burrow.net
|
||||
AWS_ACCESS_KEY_ID=<GARAGE_BACKUP_ACCESS_KEY_ID from the host env>
|
||||
AWS_SECRET_ACCESS_KEY=<GARAGE_BACKUP_SECRET_ACCESS_KEY from the host env>
|
||||
BURROW_NIX_CACHE_GCS_BUCKET=burrow-net-nix-cache
|
||||
```
|
||||
|
||||
The workflow authenticates to Google through Authentik-backed WIF and uses
|
||||
`gcloud storage rsync`. It does not use rclone or Google service-account JSON
|
||||
keys. Destination deletes are disabled by default; set
|
||||
`BURROW_GARAGE_BACKUP_DELETE=true` only for an intentional prune run.
|
||||
Loading…
Add table
Add a link
Reference in a new issue