326 lines
16 KiB
Markdown
326 lines
16 KiB
Markdown
# `BEP-0010` - Platform Services, Core, and Distribution
|
|
|
|
```text
|
|
Status: Draft
|
|
Proposal: BEP-0010
|
|
Authors: gpt-5.5
|
|
Coordinator: gpt-5.5
|
|
Reviewers: Pending
|
|
Constitution Sections: II, III, IV, V
|
|
Implementation PRs: Pending
|
|
Decision Date: Pending
|
|
```
|
|
|
|
## Summary
|
|
|
|
Burrow should grow from a forge-hosted VPN project into a coherent platform: a
|
|
KMS-backed identity and signing plane, OpenBao-backed secret distribution,
|
|
private meeting and mail services, an extracted MCP hub for operator and agent
|
|
workflows, and app builds for Apple, Linux, Android, and later store channels.
|
|
|
|
The first implementation is deliberately scaffolding-heavy. It adds OpenTofu
|
|
boundaries for Google KMS, Workload Identity Federation, GCS release backup
|
|
storage, and OpenBao; enables Garage-backed object storage and a Burrow-owned
|
|
Nix cache at `nix.burrow.net`; adds
|
|
NixOS wiring for Grafana, Prometheus, OpenTelemetry, and Jaeger; adds
|
|
disabled-by-default NixOS service switch points for `vault.burrow.net` and
|
|
`meet.burrow.net`; introduces a small cross-platform Rust core and Android
|
|
Kotlin stub; and records the distribution model before stores or public
|
|
production services are enabled.
|
|
|
|
## Motivation
|
|
|
|
- The forge now controls source and release automation, but release signing,
|
|
SAML signing, and runner cloud access still need a stronger non-exportable key
|
|
story.
|
|
- Secrets are still primarily age-managed files. That remains a good bootstrap
|
|
path, but long-running services and agents need short-lived, attributable
|
|
runtime access.
|
|
- Burrow needs mobile and desktop clients that share protocol logic instead of
|
|
repeatedly embedding platform-local behavior.
|
|
- The GTK Flatpak can be a useful desktop shell, but the actual VPN data plane
|
|
cannot be assumed to work inside an unprivileged Flatpak sandbox.
|
|
- App store, F-Droid, Flatpak, and Play Store distribution should be designed as
|
|
separate release surfaces with explicit signing, policy, and metadata gates.
|
|
|
|
## Detailed Design
|
|
|
|
- Add `infra/identity` as the OpenTofu boundary for Google Cloud identity keys
|
|
and Authentik-backed Workload Identity Federation:
|
|
- Google project id: `project-88c23ce9-918a-470a-b33`.
|
|
- Google project number: `416198671487`.
|
|
- KMS key ring: `burrow-identity`.
|
|
- non-exportable signing keys for Authentik signing, SAML CA signing, and
|
|
release signing.
|
|
- a Forgejo runner service account that can be assumed only through the
|
|
configured Authentik WIF provider and IAM bindings. Machine-to-machine
|
|
runner tokens are scoped by the Authentik WIF client ID
|
|
`google-wif.burrow.net`, with `burrow-automation` kept as the human/group
|
|
policy boundary.
|
|
- Add `infra/releases` as the OpenTofu boundary for Google Cloud
|
|
Storage-backed release backups:
|
|
- `burrow-net-releases` backs up build artifacts and Sparkle public channels.
|
|
- `burrow-net-packages` backs up signed APT, RPM, pacman, Flatpak, and Arch
|
|
repository trees.
|
|
- `burrow-net-nix-cache` backs up the Garage `attic` bucket and remains
|
|
private by default.
|
|
- Garage is the required S3-compatible primary target for release and package
|
|
uploads.
|
|
- `services.burrow.garage` is enabled by the forge host and bootstraps the
|
|
single-node layout, the `attic`, `burrow-releases`, and `burrow-packages`
|
|
buckets, scoped owner/write S3 access keys, and a read-only backup key from
|
|
a host-local environment file.
|
|
- Forgejo jobs authenticate with Authentik-backed WIF before using `gcloud
|
|
storage` for backup and Google KMS for signing; rclone and Google
|
|
service-account JSON keys are not part of the release path.
|
|
- Add `services.burrow.nixCache` for `nix.burrow.net`:
|
|
- Attic serves `https://nix.burrow.net/burrow` as the public Burrow cache.
|
|
- Attic stores chunks in the Garage `attic` bucket through the local S3 API.
|
|
- Workers use the cache only after `BURROW_NIX_CACHE_PUBLIC_KEY` is published
|
|
in Forgejo variables.
|
|
- Push access uses `/var/lib/burrow/nix-cache/ci-push-token`, sealed into
|
|
Forgejo/OpenBao as `BURROW_NIX_CACHE_PUSH_TOKEN`, instead of the host-local
|
|
bootstrap admin token.
|
|
- `.forgejo/workflows/publish-nix-cache.yml` builds selected flake outputs and
|
|
publishes their closures to Attic.
|
|
- Add `services.burrow.observability` as the forge host observability spine:
|
|
- Prometheus scrapes local system, Grafana, Headscale, OpenTelemetry
|
|
collector, and Jaeger metrics.
|
|
- Headscale exposes local metrics on `127.0.0.1:9098`.
|
|
- Tailscale SaaS metrics can be enabled later through the Prometheus
|
|
Tailscale exporter once the OAuth environment file is provisioned.
|
|
- Burrow backend/frontend processes emit OTLP to the local collector at
|
|
`127.0.0.1:4317` or `127.0.0.1:4318`.
|
|
- The collector exports traces to local Jaeger, while Grafana queries Jaeger
|
|
server-side through a provisioned datasource.
|
|
- Add `infra/openbao` as the OpenTofu boundary for OpenBao:
|
|
- Google KMS seal wrapping.
|
|
- an `age/` KV v2 mount for migrated runtime secrets.
|
|
- Authentik OIDC roles for administrators and automation.
|
|
- AppRole policy scaffolding for machine consumers.
|
|
- Add NixOS modules under `services.burrow.openbao` and
|
|
`services.burrow.jitsi`:
|
|
- both are imported by the forge host but disabled by default.
|
|
- DNS, reverse proxy, KMS seal, storage, and UDP/video bridge requirements are
|
|
visible in repo-owned options before activation.
|
|
- live enablement requires a separate deploy with secrets, DNS, and rollback
|
|
evidence.
|
|
- Keep webmail as a second mail phase:
|
|
- `inbox.burrow.net` is the intended webmail surface.
|
|
- Stalwart is the planned mail server, but the current Nixpkgs module set in
|
|
this flake does not expose the expected `services.stalwart-mail` option.
|
|
- Forward Email remains the current production mail path until a Stalwart BEP
|
|
revision names the exact NixOS module, ports, spam policy, backup target,
|
|
DKIM rotation, and migration plan.
|
|
- Extract an MCP hub to a dedicated compatible.systems repository:
|
|
- target repository: `compatible.systems/burrow/mcp-hub`.
|
|
- this repository keeps `services/mcp-hub/` as the extraction plan and
|
|
bootstrap manifest until the external repo exists.
|
|
- MCP tools should cover OpenBao, Authentik, KMS/WIF, Forgejo runners, Jitsi,
|
|
mail, release signing, and Burrow agent identity.
|
|
- Create a `burrow-core` Rust crate as the cross-platform protocol/application
|
|
core boundary:
|
|
- no direct TUN, NetworkExtension, VpnService, GTK, SwiftUI, or desktop UI
|
|
dependencies.
|
|
- platform shells link it through a thin FFI crate or native Rust API.
|
|
- target platforms include Linux x86_64/aarch64/riscv64, Android, macOS, iOS,
|
|
visionOS, and future Windows.
|
|
- Add an Android stub:
|
|
- Kotlin app shell in `Android/app`.
|
|
- Rust JNI/FFI crate in `crates/burrow-mobile-ffi`.
|
|
- the stub loads the Rust core and displays its version.
|
|
- full VPN support must use Android `VpnService`; it must not bypass platform
|
|
disclosure and consent flows.
|
|
- Keep Bazel as the stable build orchestration entrypoint:
|
|
- platform lanes expose wrapper targets under `bazel/`.
|
|
- Bazel actions may call repo-owned scripts, Cargo, Gradle, Xcode, Meson, and
|
|
OpenTofu while the project migrates deeper native rules selectively.
|
|
- Namespace cache setup remains the fast path for remote Bazel and local disk
|
|
caches.
|
|
- Use a shared Burrow owl identity across app shells:
|
|
- `design/brand/burrow-owl-icon-master.png` is the generated raster master for
|
|
Apple app icon outputs.
|
|
- Apple asset catalogs derive all `AppIcon` PNG sizes from that master and
|
|
use a restrained warm brown/gold accent color.
|
|
- GTK keeps a simple vector owl/burrow icon and symbolic icon so Linux
|
|
desktops do not inherit the old blue gear-style mark.
|
|
- Brown/black stays the core palette, with lighter brown, orange, or gold used
|
|
only as accents for eyes, beak, highlights, and platform tint.
|
|
- For Linux distribution, split the GUI package from the privileged daemon:
|
|
- DEB, RPM, pacman, AUR, and the NixOS flake/module install `burrowd`,
|
|
systemd units, D-Bus policy, and polkit policy.
|
|
- APT, RPM, and pacman repository metadata is generated from release package
|
|
artifacts and signed through Google KMS-backed OpenPGP signatures.
|
|
- Repository trees publish through the storage wrapper: Garage first when
|
|
configured, then GCS backup after the runner obtains a short-lived Google
|
|
credential from Authentik-backed WIF.
|
|
- AUR is a source-build git repository, with `burrow-git` tracked in this
|
|
repo before publication to `aur.archlinux.org`.
|
|
- Flatpak and AppImage packages are GUI-first shells that connect to the host
|
|
daemon/helper.
|
|
- Flatpak may request background/autostart permission for the GUI, but it
|
|
must not be responsible for the privileged VPN data plane.
|
|
- PackageKit may be used only as an optional native-package bootstrap on
|
|
supported distros, with narrow D-Bus permission and polkit authorization.
|
|
|
|
## Security and Operational Considerations
|
|
|
|
- Google KMS keys are non-exportable and have `prevent_destroy` enabled.
|
|
- Workload Identity Federation is scoped by issuer, audience, mapped
|
|
attributes, and service-account IAM bindings. Runners should not receive
|
|
static Google credentials.
|
|
- OpenBao bootstrap tokens and AppRole SecretIDs must never enter OpenTofu
|
|
state. OpenTofu manages mounts, policies, roles, and cloud KMS objects only.
|
|
- OpenBao should start behind `vault.burrow.net` only after:
|
|
- KMS seal key exists and decrypt permission is limited to the service
|
|
identity.
|
|
- root/admin token bootstrap is recorded out of band.
|
|
- backup/restore and seal/unseal drills are documented.
|
|
- Jitsi should start behind `meet.burrow.net` only after:
|
|
- HTTPS and XMPP domains resolve.
|
|
- UDP media ports are intentionally opened.
|
|
- Authentik guest/member policy is decided.
|
|
- Flatpak cannot be the privileged VPN backend by itself. A Flatpak GUI may talk
|
|
to a host daemon or privileged helper, but opening TUN devices, installing
|
|
routes, and owning system DNS/routing policy belong outside the sandbox.
|
|
- Host service bootstrap should be auditable. A Flatpak may discover the daemon,
|
|
open native install instructions, or request a host helper through a narrow
|
|
D-Bus name. It must not require broad system-bus access, host filesystem
|
|
access, or `flatpak-spawn --host` as the normal production path.
|
|
- Mobile VPN support uses platform VPN APIs:
|
|
- Android: `VpnService` with Play policy disclosure for Play Store builds.
|
|
- iOS and visionOS: NetworkExtension packet tunnel provider.
|
|
- Release signing, SAML CA signing, and Authentik signing should remain separate
|
|
keys even when they share a KMS key ring.
|
|
- Package repository signing should stay split by repository format. APT, RPM,
|
|
pacman, Flatpak, and AUR source surfaces each get a dedicated KMS key so a
|
|
compromised publisher path does not automatically sign every Linux channel.
|
|
- Observability ingress is local-first. Prometheus, the OpenTelemetry collector,
|
|
and Jaeger bind to loopback; Grafana is the authenticated public surface.
|
|
- Jaeger should not be exposed publicly until an Authentik-protected route or
|
|
equivalent access boundary is added.
|
|
- GCS is a backup target, not the storage abstraction. Future multi-cloud
|
|
failover should add explicit provider mirrors or S3-compatible endpoints
|
|
behind the storage wrappers.
|
|
- Garage needs real host storage for object data and metadata. It replicates
|
|
across Garage nodes, but it does not manage multiple external storage
|
|
backends. GCS and later providers should be explicit mirror/backup jobs.
|
|
- `.forgejo/workflows/backup-garage-storage.yml` is the first explicit mirror.
|
|
It uses a read-only Garage key and Authentik-backed WIF to copy Garage bucket
|
|
contents to GCS without rclone or static Google service-account keys.
|
|
|
|
## Contributor Playbook
|
|
|
|
1. Validate BEP metadata:
|
|
```bash
|
|
python3 Scripts/check-bep-metadata.py
|
|
```
|
|
2. Check the Android/Rust stub:
|
|
```bash
|
|
cargo check --locked -p burrow-mobile-ffi
|
|
bazel build //bazel/android:check_stub_stamp
|
|
```
|
|
3. Validate identity OpenTofu syntax:
|
|
```bash
|
|
Scripts/identity-tofu.sh init -backend=false
|
|
Scripts/identity-tofu.sh validate
|
|
```
|
|
4. Validate release storage OpenTofu syntax:
|
|
```bash
|
|
Scripts/releases-tofu.sh init -backend=false
|
|
Scripts/releases-tofu.sh validate
|
|
```
|
|
5. Validate observability Nix wiring and dashboard JSON:
|
|
```bash
|
|
nix eval .#nixosConfigurations.burrow-forge.config.services.prometheus.scrapeConfigs
|
|
jq empty services/grafana/dashboards/headscale.json services/grafana/dashboards/observability.json
|
|
```
|
|
6. Validate the Nix cache module wiring:
|
|
```bash
|
|
nix eval .#nixosConfigurations.burrow-forge.config.services.atticd.settings.storage --json
|
|
```
|
|
7. Validate cache publishing and Garage backup wrapper syntax:
|
|
```bash
|
|
bash -n Scripts/ci/publish-nix-cache.sh Scripts/ci/backup-garage-to-gcs.sh
|
|
```
|
|
8. Validate OpenBao OpenTofu syntax:
|
|
```bash
|
|
Scripts/openbao-tofu.sh init -backend=false
|
|
Scripts/openbao-tofu.sh validate
|
|
```
|
|
9. Before enabling `vault.burrow.net`, create or import the Google KMS seal key,
|
|
configure OpenBao bootstrap access, and record a restore test.
|
|
10. Before enabling `meet.burrow.net`, verify DNS, TLS, UDP reachability, and
|
|
Authentik access policy.
|
|
11. Before distributing Flatpak as more than a GUI, prove the host daemon/helper
|
|
path on a real Linux desktop.
|
|
12. Before Play Store distribution, complete the `VpnService` declaration and
|
|
data-safety review.
|
|
13. Before relying on PackageKit bootstrap, test Fedora, Debian/Ubuntu, and an
|
|
immutable/atomic desktop separately. PackageKit support is not universal and
|
|
should degrade to native install instructions.
|
|
14. Before promoting package repositories to stable, run the repository
|
|
publisher, import the generated public key on a clean VM for each package
|
|
family, and install Burrow from that repository rather than from the loose
|
|
artifact.
|
|
|
|
## Alternatives Considered
|
|
|
|
- Enable OpenBao and Jitsi immediately on the forge host. Rejected because both
|
|
change public ingress and security boundaries before secrets, DNS, and
|
|
rollback evidence are complete.
|
|
- Store Google service-account JSON in Forgejo secrets. Rejected because WIF
|
|
gives the runner an attributable short-lived credential path without static
|
|
cloud keys.
|
|
- Put Android directly on the existing daemon crate. Rejected for now because
|
|
the daemon carries OS-specific TUN and service assumptions. A small
|
|
cross-platform core gives mobile a safer first boundary.
|
|
- Treat Flatpak as the complete Linux VPN package. Rejected because routing,
|
|
TUN, and DNS changes need host authority.
|
|
- Use broad Flatpak host escape permissions to fork the daemon directly.
|
|
Rejected because that defeats the value of Flatpak and makes review harder
|
|
than shipping a small native daemon package with explicit polkit policy.
|
|
- Use one package signing key for every Linux repository. Rejected because
|
|
format-specific keys give cleaner revocation and narrower IAM grants.
|
|
|
|
## Impact on Other Work
|
|
|
|
- BEP-0009 release work gains identity and app-distribution follow-up lanes.
|
|
- BEP-0005 and BEP-0006 remain authoritative for daemon/client boundaries.
|
|
- BEP-0004 remains authoritative for current hosted mail until a Stalwart
|
|
revision supersedes it.
|
|
- MCP extraction should not move forge runner code until the external
|
|
compatible.systems repository exists and import paths are stable.
|
|
|
|
## Decision
|
|
|
|
Pending.
|
|
|
|
## References
|
|
|
|
- `CONSTITUTION.md`
|
|
- `contributors.nix`
|
|
- `evolution/proposals/BEP-0004-hosted-mail-and-saas-identity.md`
|
|
- `evolution/proposals/BEP-0005-daemon-ipc-and-apple-boundary.md`
|
|
- `evolution/proposals/BEP-0006-tailnet-authority-first-control-plane.md`
|
|
- `evolution/proposals/BEP-0009-release-infrastructure-and-store-upload-pipeline.md`
|
|
- `infra/identity/`
|
|
- `infra/releases/`
|
|
- `infra/openbao/`
|
|
- `docs/OBSERVABILITY.md`
|
|
- `docs/NIX_CACHE.md`
|
|
- `.forgejo/workflows/publish-nix-cache.yml`
|
|
- `.forgejo/workflows/backup-garage-storage.yml`
|
|
- `Scripts/ci/publish-nix-cache.sh`
|
|
- `Scripts/ci/backup-garage-to-gcs.sh`
|
|
- `package/repositories/`
|
|
- `nixos/modules/burrow-garage.nix`
|
|
- `nixos/modules/burrow-nix-cache.nix`
|
|
- `nixos/modules/burrow-observability.nix`
|
|
- `nixos/modules/burrow-openbao.nix`
|
|
- `nixos/modules/burrow-jitsi.nix`
|
|
- `crates/burrow-core/`
|
|
- `crates/burrow-mobile-ffi/`
|
|
- `Android/`
|
|
- `bazel/android/`
|
|
- `services/mcp-hub/`
|