Wire Forge-native release infrastructure
Some checks failed
Cache: Publish Nix / Publish Nix Cache (push) Waiting to run
Build Rust / Cargo Test (push) Successful in 4m14s
Build Site / Next.js Build (push) Failing after 6s
Infra: OpenTofu / OpenTofu (grafana) (push) Successful in 5s
Lint Governance / BEP Metadata (push) Successful in 0s
Build: Android / Android Rust Core Stub (push) Failing after 23s
Some checks failed
Cache: Publish Nix / Publish Nix Cache (push) Waiting to run
Build Rust / Cargo Test (push) Successful in 4m14s
Build Site / Next.js Build (push) Failing after 6s
Infra: OpenTofu / OpenTofu (grafana) (push) Successful in 5s
Lint Governance / BEP Metadata (push) Successful in 0s
Build: Android / Android Rust Core Stub (push) Failing after 23s
This commit is contained in:
parent
97c569fb35
commit
002bd382e9
199 changed files with 14268 additions and 185 deletions
60
docs/OBSERVABILITY.md
Normal file
60
docs/OBSERVABILITY.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
# Burrow Observability
|
||||
|
||||
The forge host imports `services.burrow.observability`, which wires the first
|
||||
production observability spine:
|
||||
|
||||
- Prometheus on `127.0.0.1:9090`
|
||||
- OpenTelemetry collector OTLP gRPC on `127.0.0.1:4317`
|
||||
- OpenTelemetry collector OTLP HTTP on `127.0.0.1:4318`
|
||||
- collector Prometheus export on `127.0.0.1:9464`
|
||||
- Jaeger all-in-one on local ports only
|
||||
- Grafana at `graphs.burrow.net`, backed by Authentik SSO
|
||||
|
||||
Grafana datasources are provisioned by NixOS:
|
||||
|
||||
- `prometheus` points at local Prometheus
|
||||
- `jaeger` points at the local Jaeger query API
|
||||
|
||||
OpenTofu owns checked-in dashboards under `services/grafana/dashboards/`:
|
||||
|
||||
- `burrow-overview.json`
|
||||
- `headscale.json`
|
||||
- `observability.json`
|
||||
|
||||
## Metrics
|
||||
|
||||
Prometheus scrapes:
|
||||
|
||||
- Prometheus self metrics
|
||||
- node exporter
|
||||
- systemd exporter
|
||||
- Grafana `/metrics`
|
||||
- Headscale `/metrics` on `127.0.0.1:9098`
|
||||
- OpenTelemetry collector metrics
|
||||
- Jaeger admin metrics
|
||||
- Garage admin metrics when `services.burrow.garage.enable = true`
|
||||
- the optional Tailscale exporter when its OAuth environment file is configured
|
||||
|
||||
Headscale and Tailscale are treated as the same Tailnet family in dashboards.
|
||||
Headscale exposes a local Prometheus listener through
|
||||
`services.burrow.headscale.enableMetrics`. Tailscale SaaS metrics require the
|
||||
NixOS Tailscale exporter and an OAuth environment file; the module leaves that
|
||||
disabled until the credential is provisioned.
|
||||
|
||||
## Traces
|
||||
|
||||
Burrow services that emit OpenTelemetry traces should use:
|
||||
|
||||
```sh
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4317
|
||||
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
|
||||
OTEL_SERVICE_NAME=burrow
|
||||
```
|
||||
|
||||
The collector exports traces to Jaeger. Jaeger is intentionally local-only for
|
||||
now; Grafana can query it server-side through the provisioned datasource.
|
||||
|
||||
Headscale and Tailscale currently enter the spine through Prometheus metrics and
|
||||
systemd state. If an upstream Tailnet component starts emitting OTLP, point it
|
||||
at the same local collector endpoint rather than exposing Jaeger or Prometheus
|
||||
directly.
|
||||
Loading…
Add table
Add a link
Reference in a new issue