Wire Forge-native release infrastructure
Some checks failed
Cache: Publish Nix / Publish Nix Cache (push) Waiting to run
Build Rust / Cargo Test (push) Successful in 4m14s
Build Site / Next.js Build (push) Failing after 6s
Infra: OpenTofu / OpenTofu (grafana) (push) Successful in 5s
Lint Governance / BEP Metadata (push) Successful in 0s
Build: Android / Android Rust Core Stub (push) Failing after 23s

This commit is contained in:
Conrad Kramer 2026-06-07 05:51:12 -07:00
parent 97c569fb35
commit 002bd382e9
199 changed files with 14268 additions and 185 deletions

130
docs/DISTRIBUTION.md Normal file
View file

@ -0,0 +1,130 @@
# Distribution
Burrow distribution is split by platform authority.
## Apple
macOS, iOS, and visionOS builds use the Apple app plus NetworkExtension packet
tunnel provider. App Store and TestFlight upload lanes stay separate from build
lanes so signing, export, metadata, and external distribution are visible.
Developer ID signing is moving to a KMS-backed key path. The
`apple-developer-id-application` key in the Burrow `burrow-identity` Google KMS
key ring is non-exportable and can produce a standard Apple CSR through
`Scripts/apple/google-kms-csr.py` or the manual Forgejo
`apple-developer-id-kms-csr.yml` workflow. The active G2 Developer ID
Application certificate is `9JKN6HXBHC`, stored as public material at
`Apple/Certificates/developer-id-application-9JKN6HXBHC.cer`, and expires on
`2031-06-08`.
App Store iOS signing uses the same non-exportable model. The
`apple-ios-distribution` key in the Burrow identity key ring produced the active
`IOS_DISTRIBUTION` certificate `3G42677598`, stored as public material at
`Apple/Certificates/ios-distribution-3G42677598.cer`, and expires on
`2027-06-07`. App Store Connect certificate creation can be driven by
`Scripts/apple/create-asc-certificate.mjs` once a KMS-backed CSR has been
generated.
When `BURROW_IOS_DISTRIBUTION_CERTIFICATE_ID` is set, provisioning-profile sync
pins iOS App Store profiles to that certificate. Existing App Store profiles
with the same name are deleted and recreated if Apple reports that they contain
a different distribution certificate.
Sparkle appcast signing uses the non-exportable `sparkle-ed25519` Google KMS
key. macOS release builds embed public EdDSA key
`Myv9ZNZT6YGKMtMezh52ra4WqaeEKc4VlvVU0evhJeI=` and point Sparkle metadata at
`https://releases.burrow.net/sparkle/appcast.xml`. The appcast signer runs only
when `BURROW_SPARKLE_SIGN_WITH_KMS=true` so unsigned local validation artifacts
do not require Google credentials.
This is not equivalent to the existing `.p12` keychain import path. The current
Xcode packaging lane can still use exportable signing assets, while the
KMS-backed Apple certificates require a follow-up signer integration that can
ask Google KMS to produce code signatures without exporting the private key.
## Android
Android starts with a Kotlin shell and Rust core FFI stub. Full VPN support must
use Android `VpnService`, which means Play Store builds need the VPN service
declaration, clear in-app disclosure, and data-safety review before upload.
## Linux
The GTK app is the desktop UI. For VPN operation, it should talk to a host
daemon or privileged helper that owns TUN, routes, DNS, and firewall state.
Flatpak can package the GUI, but it should not be treated as the privileged data
plane. The preferred Linux shape is:
- native packages install `burrowd`, the systemd unit/socket, D-Bus policy, and
polkit policy
- the Flatpak GUI talks to the host service over a narrowly named D-Bus API or a
constrained local daemon socket
- the daemon performs privileged operations only after polkit authorizes the
user action
A Flatpak build can either:
- connect to a system/user Burrow daemon over a constrained local API
- ask a host-installed helper to start or configure the daemon through a
reviewed D-Bus policy
- stay as a configuration/status UI when no helper is installed
It should not silently install native packages. PackageKit can be an optional
bootstrap path on distros where the native Burrow daemon package is available,
PackageKit is installed, and the Flatpak has a narrow
`org.freedesktop.PackageKit` system-bus permission. That path is a convenience,
not the baseline, because it is distro-dependent and still requires native
package trust and polkit authorization.
The native repository plan is now:
- `packages.burrow.net/apt` for Debian-family packages
- `packages.burrow.net/rpm/<channel>/<basearch>` for RPM-family packages
- `packages.burrow.net/arch/burrow/<arch>` for pacman binary packages
- `aur.archlinux.org/burrow-git.git` for the Arch source-build package
- a Flatpak repository descriptor for the GTK GUI, paired with the native daemon
- the NixOS flake/module for declarative NixOS installs
Package repository signing is split by repository format. Google Cloud KMS owns
non-exportable RSA signing keys for APT, RPM, pacman, Flatpak, and AUR source
surfaces. The current repository builder signs APT `Release`, RPM `repomd.xml`,
and pacman database/package sidecar signatures through the KMS OpenPGP bridge.
Embedded RPM package signatures and Flatpak OSTree summary signing still need a
dedicated implementation pass before those channels are promoted beyond
staging.
Release artifacts and signed repository trees publish through repo-owned storage
wrappers. Garage is the required S3-compatible primary target; GCS remains the
first backup target through buckets owned by `infra/releases`:
`burrow-net-releases` for build artifacts and `burrow-net-packages` for package
repositories. CI authenticates with Authentik-backed Google Workload Identity
Federation before using Google KMS or `gcloud storage`; no Google
service-account JSON key or rclone transport is part of the release path.
An AppImage may be useful as a portable GUI on distributions without current
Flatpak support, but it has the same daemon boundary as Flatpak. NixOS should
prefer the flake/module path so daemon, policy, and service activation are
declarative.
## F-Droid and Play Store
F-Droid should build from source with reproducible metadata and no proprietary
Play dependencies in the free flavor. Play Store can use the same Rust core but
requires the Play policy path for `VpnService`.
## Flatpak
Flatpak distribution is useful for desktop reach, but the app sandbox does not
grant route, DNS, or TUN authority by itself. Burrow should publish Flatpak only
with clear daemon/helper requirements until a real desktop smoke test proves the
full flow.
Flatpak can start child processes inside its sandbox and can request background
or autostart permission for the Flatpak app. That is not enough for the VPN data
plane. Starting the host daemon belongs to one of these paths:
- systemd socket or D-Bus activation installed by the native package
- explicit D-Bus call to a host helper with narrow permissions and polkit
- PackageKit-assisted native package install where supported
- manual native package install instructions opened from the GUI

View file

@ -75,7 +75,7 @@ rustup toolchain install stable-msvc
1. Clone the repository:
```
git clone git@github.com:hackclub/burrow.git
git clone https://git.burrow.net/hackclub/burrow.git
```
2. Open the `burrow` folder in Visual Studio Code:

View file

@ -1,7 +1,10 @@
# Linux GTK App Getting Started
Currently, the GTK App can be built as a binary or as an AppImage.
Note that the flatpak version can compile but will not run properly!
Currently, the GTK App can be built as a binary or as an AppImage. The Flatpak
lane is a GUI packaging target until Burrow has a verified host daemon/helper
path for TUN, routes, DNS, and firewall state. The intended Flatpak behavior is
to discover and talk to a native Burrow daemon installed by DEB, RPM, or the
NixOS flake/module, not to own privileged routing itself.
## Dependencies

88
docs/NIX_CACHE.md Normal file
View file

@ -0,0 +1,88 @@
# Burrow Nix Cache
`nix.burrow.net` is the Burrow Nix binary cache surface for forge jobs and
workers.
The host layout is:
- Garage stores objects on the forge host's Garage data volume.
- Attic serves the Nix binary cache API at `nix.burrow.net`.
- Attic stores NAR chunks in the Garage `attic` bucket through the local S3 API.
- Workers consume `https://nix.burrow.net/burrow` as a substituter once the
cache public key is configured.
Garage is not a multi-backend object sync manager. It can replicate across
Garage nodes, but it does not write the same object to GCS and another provider
as independent storage backends. Multi-cloud backup should be a separate mirror
or export job from Garage buckets to GCS and later to additional providers.
## Host Bootstrap
`services.burrow.garage` creates a host-local environment file at
`/var/lib/burrow/garage/env`. The file is not stored in Git or the Nix store.
It contains:
- Garage RPC secret
- Garage access keys for Attic, release artifacts, package repositories, and
read-only backups
- Attic JWT signing secret
- AWS-compatible variables used by Attic to talk to Garage
`burrow-garage-bootstrap.service` creates the initial single-node Garage layout,
imports the generated keys, and creates the `attic`, `burrow-releases`, and
`burrow-packages` buckets. It grants the backup key read-only access to those
three buckets so CI can mirror them without receiving owner/write access to
Garage.
`services.burrow.nixCache` enables Attic, points it at Garage, exposes it through
Caddy, creates a public `burrow` cache, and writes two host-local tokens:
- `/var/lib/burrow/nix-cache/admin-token`: bootstrap/admin recovery token
- `/var/lib/burrow/nix-cache/ci-push-token`: scoped pull/push token for the
`burrow` cache
## Worker Configuration
`Scripts/ci/ensure-nix.sh` uses the Burrow cache only when
`BURROW_NIX_CACHE_PUBLIC_KEY` is present:
```sh
BURROW_NIX_CACHE_URL=https://nix.burrow.net/burrow
BURROW_NIX_CACHE_PUBLIC_KEY=<cache public key from attic cache info>
```
Without the public key, CI falls back to `cache.nixos.org` only. This avoids
configuring an unsigned or untrusted substituter.
After deployment, get the public key from the host:
```sh
attic cache info local:burrow
```
Then set `BURROW_NIX_CACHE_PUBLIC_KEY` in Forgejo variables.
Seal the scoped push token from `/var/lib/burrow/nix-cache/ci-push-token` into
Forgejo/OpenBao as `BURROW_NIX_CACHE_PUSH_TOKEN`. The
`Cache: Publish Nix` workflow runs `Scripts/ci/publish-nix-cache.sh`, builds the
selected flake outputs, and pushes their closures to Attic. It skips cleanly
when the push token is absent unless `BURROW_NIX_CACHE_REQUIRED=true`.
## Backups
The Garage `attic` bucket is backed up to the private GCS bucket
`burrow-net-nix-cache`. Release artifacts and package repositories are backed up
to their public GCS buckets. The scheduled `Backup: Garage Storage` workflow
runs `Scripts/ci/backup-garage-to-gcs.sh` with:
```sh
BURROW_GARAGE_ENDPOINT=https://objects.burrow.net
AWS_ACCESS_KEY_ID=<GARAGE_BACKUP_ACCESS_KEY_ID from the host env>
AWS_SECRET_ACCESS_KEY=<GARAGE_BACKUP_SECRET_ACCESS_KEY from the host env>
BURROW_NIX_CACHE_GCS_BUCKET=burrow-net-nix-cache
```
The workflow authenticates to Google through Authentik-backed WIF and uses
`gcloud storage rsync`. It does not use rclone or Google service-account JSON
keys. Destination deletes are disabled by default; set
`BURROW_GARAGE_BACKUP_DELETE=true` only for an intentional prune run.

60
docs/OBSERVABILITY.md Normal file
View file

@ -0,0 +1,60 @@
# Burrow Observability
The forge host imports `services.burrow.observability`, which wires the first
production observability spine:
- Prometheus on `127.0.0.1:9090`
- OpenTelemetry collector OTLP gRPC on `127.0.0.1:4317`
- OpenTelemetry collector OTLP HTTP on `127.0.0.1:4318`
- collector Prometheus export on `127.0.0.1:9464`
- Jaeger all-in-one on local ports only
- Grafana at `graphs.burrow.net`, backed by Authentik SSO
Grafana datasources are provisioned by NixOS:
- `prometheus` points at local Prometheus
- `jaeger` points at the local Jaeger query API
OpenTofu owns checked-in dashboards under `services/grafana/dashboards/`:
- `burrow-overview.json`
- `headscale.json`
- `observability.json`
## Metrics
Prometheus scrapes:
- Prometheus self metrics
- node exporter
- systemd exporter
- Grafana `/metrics`
- Headscale `/metrics` on `127.0.0.1:9098`
- OpenTelemetry collector metrics
- Jaeger admin metrics
- Garage admin metrics when `services.burrow.garage.enable = true`
- the optional Tailscale exporter when its OAuth environment file is configured
Headscale and Tailscale are treated as the same Tailnet family in dashboards.
Headscale exposes a local Prometheus listener through
`services.burrow.headscale.enableMetrics`. Tailscale SaaS metrics require the
NixOS Tailscale exporter and an OAuth environment file; the module leaves that
disabled until the credential is provisioned.
## Traces
Burrow services that emit OpenTelemetry traces should use:
```sh
OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4317
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
OTEL_SERVICE_NAME=burrow
```
The collector exports traces to Jaeger. Jaeger is intentionally local-only for
now; Grafana can query it server-side through the provisioned datasource.
Headscale and Tailscale currently enter the spine through Prometheus metrics and
systemd state. If an upstream Tailnet component starts emitting OTLP, point it
at the same local collector endpoint rather than exposing Jaeger or Prometheus
directly.

99
docs/PLATFORM_SERVICES.md Normal file
View file

@ -0,0 +1,99 @@
# Platform Services
Burrow's next platform layer is split into service boundaries that can be
enabled independently after their security and rollback checks pass.
## Identity and KMS
`infra/identity` owns Google Cloud KMS and Workload Identity Federation
resources for the Burrow Google project:
- project id: `project-88c23ce9-918a-470a-b33`
- project number: `416198671487`
- key ring: `burrow-identity`
- intended keys: Authentik signing, SAML CA signing, release signing
The WIF path is for Authentik-backed Forgejo runners. Runners should exchange an
OIDC token for a short-lived Google credential instead of carrying static
service-account JSON.
Release storage uses provider-neutral upload wrappers. Garage is the required
S3-compatible primary target. GCS is the first backup target and remains the
compatibility source for current
Sparkle/store follow-on jobs:
- release artifacts: `burrow-net-releases`
- signed package repositories: `burrow-net-packages`
- private Nix cache backup: `burrow-net-nix-cache`
Forgejo jobs authenticate to the same Google project through Authentik-backed
WIF, then use `gcloud storage` for upload/download and Google KMS for release
and repository signatures. No rclone transport or Google service-account JSON
key is part of the release path.
`services.burrow.garage` is enabled by the forge host. It provides the host
activation point for Garage API, static-web, and admin listeners, and
bootstraps the initial single-node layout plus the `attic`, `burrow-releases`,
and `burrow-packages` buckets. It creates separate owner/write keys for Attic,
release artifacts, and package repositories, plus a read-only backup key for
the scheduled Garage-to-GCS mirror.
Garage is now the primary object-storage surface for release/package uploads and
the Nix cache. It uses host-local storage for object data and metadata; it does
not fan out writes to multiple object-store backends. Multi-cloud failover
should be implemented as explicit Garage bucket mirrors/backups, with GCS as the
first backup target.
## Nix Cache
`nix.burrow.net` is the Burrow Nix binary cache surface. The forge host runs
Attic behind Caddy and stores cache chunks in the Garage `attic` bucket through
Garage's local S3 API. Workers can use `https://nix.burrow.net/burrow` once the
cache public key is published as `BURROW_NIX_CACHE_PUBLIC_KEY` in Forgejo. The
host bootstrap also creates `/var/lib/burrow/nix-cache/ci-push-token`; seal it
as `BURROW_NIX_CACHE_PUSH_TOKEN` before enabling broad cache publication.
## Observability
`graphs.burrow.net` is backed by Grafana with Authentik SSO. The forge host now
imports `services.burrow.observability`, which starts local Prometheus,
OpenTelemetry collector, and Jaeger services. NixOS provisions Grafana
datasources; OpenTofu manages checked-in dashboards for the Burrow overview,
Headscale, and the broader Prometheus/OpenTelemetry/Jaeger spine.
Jaeger is local-only at first and is queried through Grafana. Headscale metrics
are scraped locally. Tailscale SaaS metrics are optional until the OAuth
credential file exists.
## OpenBao
`vault.burrow.net` is the planned OpenBao surface. The repository now has:
- `infra/openbao` for KMS seal wrapping and OpenBao auth/policy resources
- `services.burrow.openbao` as a disabled-by-default NixOS switch point
- `Scripts/openbao-tofu.sh` for stack-local OpenTofu operations
OpenBao should not be enabled until the KMS seal key, bootstrap token handling,
backup/restore procedure, and Authentik roles are verified.
## Jitsi Meet
`meet.burrow.net` is the planned Jitsi Meet surface. The repository now has
`services.burrow.jitsi` as a disabled-by-default NixOS switch point.
Jitsi is not just an HTTP service. The rollout needs DNS, TLS, XMPP/prosody
state, and an explicit UDP media-port decision before activation.
## Mail and Webmail
`inbox.burrow.net` is the intended webmail surface. Stalwart is the planned mail
server, but this flake does not currently expose the expected Stalwart NixOS
option. Forward Email remains the current production mail path until the
Stalwart module name, ports, DKIM rotation, backup target, spam policy, and
migration steps are pinned in a follow-up BEP update.
## MCP Hub
The MCP hub should be extracted to `compatible.systems/burrow/mcp-hub`. Until
that repository exists, `services/mcp-hub/` records the extraction boundary and
tool inventory.