Add proxy subscription runtime support
Add daemon RPCs, Apple and GTK import flows, packet proxy runtime support, diagnostics, and BEPs for proxy subscription handling. Redact subscription URL secrets from fetch errors before they reach logs or UI surfaces.
This commit is contained in:
JettChenT
parent
97c569fb35
commit
d1638726ca
46 changed files with 15079 additions and 456 deletions
|
|
@ -0,0 +1,271 @@
|
|||
# `BEP-0010` - Proxy Subscriptions as Packet Tunnel Networks
|
||||
|
||||
```text
|
||||
Status: Draft
|
||||
Proposal: BEP-0010
|
||||
Authors: Codex
|
||||
Coordinator: Codex
|
||||
Reviewers: Pending
|
||||
Constitution Sections: II, IV, V
|
||||
Implementation PRs: Pending
|
||||
Decision Date: Pending
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
Burrow should support Trojan and Shadowsocks subscription imports as packet-tunnel networks, not as local SOCKS or system proxy mode. The user-facing add action should be named `Import Proxy Subscription` on every UI surface. Imported proxy subscriptions should participate in Burrow's existing tunnel model: platform tunnel capture feeds daemon-owned routing and proxy execution, and a selected proxy outbound carries TCP and UDP flows.
|
||||
|
||||
This proposal supersedes `BEP-0009`. The earlier SOCKS-backed Trojan runtime was useful as a smoke-test path, but Burrow's product semantics are VPN-shaped. Keeping a local SOCKS mode would add another behavior surface, confuse how proxy networks relate to WireGuard and Tailnet, and delay the packet-tunnel work Burrow actually needs.
|
||||
|
||||
## Motivation
|
||||
|
||||
- Users expect the main Burrow tunnel switch to affect full-device traffic, as it does for packet-shaped WireGuard and Tailnet integrations.
|
||||
- Trojan and Shadowsocks subscriptions are commonly distributed as Clash/Mihomo YAML, proxy-provider content, plain URI lists, or base64 URI lists. Burrow's current Trojan parser only covers a narrow `trojan://` URI-list subset.
|
||||
- Mihomo's TUN mode is the closest reference behavior: it captures packets, runs a userspace IP stack, hijacks DNS, resolves rules and proxy selections, then dials protocol-specific outbounds.
|
||||
- Burrow should avoid a second user-facing "proxy mode" based on `HTTP_PROXY`, SOCKS settings, or a loopback listener because that is not equivalent to VPN behavior and creates platform-specific state outside Burrow's tunnel contract.
|
||||
- Subscription URLs and proxy node credentials are secret-bearing. Fetching, parsing, storing, refreshing, and redacting them belong in the daemon boundary described by `BEP-0005`.
|
||||
|
||||
## Detailed Design
|
||||
|
||||
### User Model
|
||||
|
||||
- Add a UI action named `Import Proxy Subscription`.
|
||||
- Imported subscriptions are stored as `Network` records because they represent routeable connectivity sources.
|
||||
- Proxy nodes inside a subscription are selectable endpoints within that network.
|
||||
- Node selection should mirror Mihomo selector behavior in `adapter/outboundgroup/selector.go`: persist an explicit selected node when the user chooses one, validate that it exists, and fall back to the first runtime-supported node only when no explicit supported selection is available.
|
||||
- Proxy subscriptions do not create `Account` records by default. Accounts remain for identity and sign-in state such as Tailnet login or future provider identity flows.
|
||||
- Apple SwiftUI, Linux GTK, and future UI surfaces should expose the same import, preview, warning, selection, refresh, and node status behavior over the daemon API.
|
||||
- UI clients may show subscription name, refresh state, selected node, node health, warnings, and import preview results. They must not fetch subscription URLs or open Trojan/Shadowsocks sockets directly.
|
||||
- Import preview UIs must expose the runtime-supported Trojan and Shadowsocks nodes as a selectable list and pass the selected ordinal to the daemon `ApplyImport` request. Parsed-but-unsupported nodes should remain visible as warnings or counts, not selectable runtime choices.
|
||||
|
||||
### Removed SOCKS Mode
|
||||
|
||||
- Remove the daemon runtime path that starts a stored Trojan profile as a local SOCKS5 proxy.
|
||||
- Remove CLI and smoke entrypoints whose primary purpose is operating Burrow as a SOCKS proxy.
|
||||
- Keep parser tests and sanitized live-comparison tooling only when they validate subscription import or packet-tunnel behavior.
|
||||
- Do not expose system `HTTP_PROXY`, system SOCKS, PAC, or "set system proxy" behavior as part of this proposal.
|
||||
- If a temporary developer-only loopback proxy is needed for debugging, it must be clearly separate from product behavior and must not be the stored network runtime.
|
||||
|
||||
### Subscription Import Compatibility
|
||||
|
||||
Implement the importer in the daemon and mirror Mihomo's behavior as closely as Burrow's narrower protocol scope allows. Path references in this section are relative to the Mihomo repository root. When behavior is ambiguous, contributors should inspect the referenced Mihomo code path and either match it or document a deliberate deviation in this BEP before implementation.
|
||||
|
||||
- Full config parsing should recognize subscription-relevant top-level fields analogous to `proxies`, `proxy-providers`, and `proxy-groups` in `config/config.go`.
|
||||
- Provider parsing should support `file`, `http`, and `inline` style sources analogous to `adapter/provider/parser.go`.
|
||||
- Provider content parsing should first try YAML with a `proxies` field. If YAML cannot be parsed, fall back to V2Ray-style URI conversion, matching the shape in `adapter/provider/provider.go`.
|
||||
- Plain and base64 URI-list inputs should follow Mihomo's behavior in `common/convert/converter.go` and `common/convert/base64.go`: attempt raw/std base64 decoding, otherwise treat the body as plaintext.
|
||||
- Trojan URI conversion should closely follow Mihomo's `trojan` case in `common/convert/converter.go` and the accepted outbound fields in `adapter/outbound/trojan.go`. Burrow should preserve Mihomo-compatible names and semantics internally where practical, then adapt them into Burrow's storage model. It should map at least:
|
||||
- URI fragment to display name
|
||||
- host, port, and username to server, port, and password
|
||||
- `allowInsecure` to certificate verification policy
|
||||
- `sni`, `alpn`, `type=ws`, `type=grpc`, client fingerprint, and pinned certificate fingerprint where supported
|
||||
- Mihomo's default client fingerprint behavior where applicable
|
||||
- Mihomo's ws and grpc option shapes before translating them into Burrow-owned structs
|
||||
- Shadowsocks URI conversion should map at least:
|
||||
- legacy base64 host form
|
||||
- base64 `cipher:password` userinfo
|
||||
- server, port, cipher, password, and node name
|
||||
- `udp-over-tcp` or `uot`
|
||||
- common `obfs` and `v2ray-plugin` plugin options when support is implemented
|
||||
- Unsupported protocols, unsupported plugin modes, malformed nodes, duplicate node names, and insecure TLS flags must be returned in import preview warnings without logging credentials.
|
||||
|
||||
Burrow should not claim full Clash or full Mihomo compatibility in this proposal. The compatibility target is subscription import for Trojan and Shadowsocks nodes plus the minimum provider and group metadata needed to select and refresh outbounds. Within that target, subscription parsing and Trojan normalization should be intentionally Mihomo-like so a subscription that imports cleanly in Mihomo has the same node set, labels, security warnings, and supported Trojan transport metadata in Burrow unless the BEP records a specific exception.
|
||||
|
||||
### Data Model
|
||||
|
||||
Add a proxy-subscription network payload with:
|
||||
|
||||
- subscription display name
|
||||
- redacted subscription URL or a credential reference to the full URL
|
||||
- provider format detected during import
|
||||
- refresh policy and last refresh result
|
||||
- ordered proxy nodes
|
||||
- selected node or selection strategy
|
||||
- certificate and transport warnings
|
||||
- per-node protocol type, server metadata, transport metadata, and secret references
|
||||
|
||||
Proxy node secrets must move out of plain network payload blobs when Burrow's secret-storage path exists. Until then, payload storage must be treated as sensitive and logs/tests must use redacted fixtures.
|
||||
|
||||
### Daemon APIs
|
||||
|
||||
Add daemon gRPC APIs before platform UI work:
|
||||
|
||||
- preview import from subscription URL
|
||||
- apply selected preview result as a network
|
||||
- refresh an existing subscription network
|
||||
- list nodes for a proxy subscription network
|
||||
- set selected node or selection strategy
|
||||
|
||||
Preview responses should include compatible count, rejected count, detected format, sanitized warnings, suggested network name, and redacted node labels. Apply and refresh must preserve local node order and selected-node state where possible.
|
||||
|
||||
### Packet Tunnel Runtime
|
||||
|
||||
Burrow should route proxy subscriptions through the existing tunnel architecture:
|
||||
|
||||
1. A platform tunnel provider starts the tunnel and applies daemon-provided network settings.
|
||||
2. Platform packet capture forwards packets through the daemon packet interface. Apple uses `PacketTunnelProvider` and the existing `TunnelPackets` gRPC stream against the packet-tunnel daemon socket owned by the Network Extension; Linux should use the daemon-owned TUN path already aligned with the GTK client.
|
||||
3. The daemon owns a proxy packet engine for active proxy-subscription networks.
|
||||
4. The packet engine reconstructs TCP and UDP flows from IP packets.
|
||||
5. DNS packets are handled by a daemon-owned DNS path so hostname metadata can be preserved for rule selection and proxy dialing.
|
||||
6. Each flow is matched to the selected proxy node or selection strategy.
|
||||
7. The daemon opens a Trojan or Shadowsocks outbound connection and relays flow bytes.
|
||||
8. Return packets are serialized back to Apple through `TunnelPackets`.
|
||||
|
||||
Mihomo's equivalent kernel path is:
|
||||
|
||||
- TUN recreation and listener lifecycle in `listener/listener.go`
|
||||
- TUN option construction and stack startup in `listener/sing_tun/server.go`
|
||||
- DNS hijack in `listener/sing_tun/dns.go`
|
||||
- metadata conversion in `listener/sing/sing.go`
|
||||
- TCP and UDP handling in `tunnel/tunnel.go`
|
||||
- outbound protocol construction in `adapter/parser.go`, `adapter/outbound/trojan.go`, and `adapter/outbound/shadowsocks.go`
|
||||
|
||||
Burrow should use these as architecture references, not as code to vendor blindly. The Burrow implementation must fit the daemon IPC boundary, Rust runtime, Apple NetworkExtension path, Linux daemon/TUN path, GTK client, and existing network selection model.
|
||||
|
||||
### Implemented Runtime Shape
|
||||
|
||||
The first packet-tunnel implementation follows the Mihomo adapter shape but keeps Burrow's narrower protocol surface:
|
||||
|
||||
- `burrow/src/proxy_runtime.rs` owns the daemon packet engine.
|
||||
- Apple uses the existing `TunnelPackets` stream, matching the Tailnet packet-stream boundary.
|
||||
- Linux uses the daemon TUN bridge path.
|
||||
- Tunnel configuration excludes resolved selected proxy server host routes from the captured default route so proxy-server dials cannot be trapped by the full tunnel.
|
||||
- On Apple, the proxy packet runtime binds Trojan and Shadowsocks outbound sockets to a non-tunnel physical interface before connect and also installs resolved Network Extension excluded routes for the selected proxy server. Loopback proxy-server addresses are not physical-interface bound so controlled local tests and developer-only local endpoints can still connect. The excluded route is intentionally redundant: it prevents proxy-server dials and server DNS artifacts from recursively entering the full tunnel when socket binding is unavailable, delayed, or ignored by a platform path.
|
||||
- On Apple, node selection and refresh are applied to both the app-facing control daemon and, while connected, the packet-tunnel daemon over `burrow-packet-tunnel.sock`. The packet daemon swaps the active proxy outbound inside the existing packet bridge, so new TCP and UDP flows use the selected node without restarting the Network Extension. Existing flows may continue on the outbound they opened with until they close.
|
||||
- TCP and UDP packets are reconstructed with the existing smoltcp userspace stack.
|
||||
- Trojan TCP nodes are dialed over TLS and use Trojan TCP and UDP request framing directly.
|
||||
- Shadowsocks AEAD nodes are dialed directly for TCP and UDP using SIP004-style salt, HKDF subkeys, and AEAD chunks/datagrams.
|
||||
- Unsupported runtime transports such as Trojan `ws`/`grpc`, Shadowsocks plugins, and Shadowsocks UDP-over-TCP are still parsed and preserved from subscriptions, but they are rejected by runtime selection until those adapter transports are implemented. If no node is explicitly selected, Burrow chooses the first runtime-supported node instead of starting an unsupported transport.
|
||||
- Apple SwiftUI and Linux GTK import flows preview nodes, restrict the picker to runtime-supported nodes, and pass the chosen node ordinal to the daemon when applying an import.
|
||||
- Proxy-subscription tunnel settings point DNS at the daemon-owned tunnel address. The packet runtime answers ordinary A queries with fake IPs and records the fake-IP to hostname mapping so later TCP flows can send hostname-form proxy requests. This avoids making website DNS depend on outbound UDP support while preserving hostname metadata for Trojan and Shadowsocks dialing.
|
||||
|
||||
### Routing, DNS, and Apple Runtime Ownership
|
||||
|
||||
For full-tunnel proxy networks, daemon tunnel configuration should include:
|
||||
|
||||
- a virtual IPv4 address from a documentation or carrier-grade NAT range suitable for the packet engine
|
||||
- a virtual IPv6 address once the packet engine is enabled for IPv6 flows
|
||||
- MTU selected for userspace packet handling
|
||||
- default IPv4 route when full-tunnel is enabled
|
||||
- default IPv6 route when the platform tunnel surface supports it
|
||||
- a daemon-owned DNS server address reachable inside the tunnel
|
||||
|
||||
DNS handling should be explicit. Proxy-subscription tunnel configuration should use the daemon-owned DNS address inside the tunnel, not ambient physical resolvers and not a public upstream that requires outbound UDP before TCP can start. The daemon DNS path may answer with fake IPs and must keep those mappings scoped to the active proxy runtime so hostname-preserving proxy dialing does not leak between networks.
|
||||
|
||||
On Apple, the app-facing daemon and packet-tunnel daemon are intentionally separate in-process daemon instances over different app-group Unix sockets. The app-facing daemon handles subscription preview, import, refresh, node selection, network list, and account/login control through `burrow.sock`. The Network Extension hosts the packet-tunnel daemon over `burrow-packet-tunnel.sock` while the VPN session is active; `TunnelStart`, `TunnelConfiguration`, `TunnelPackets`, and `TunnelStop` for the active session use that socket.
|
||||
|
||||
This split keeps macOS packet runtime ownership aligned with `NEPacketTunnelProvider`: routes, DNS settings, packet stream lifetime, and proxy runtime are created from the same selected network state when the extension starts. UI mutations persist desired state in the shared app-group database. If a mutation changes active packet-tunnel behavior, the Apple client should update the packet daemon's desired state and only restart or reassert the Network Extension when the platform tunnel settings themselves change.
|
||||
|
||||
Proxy-subscription routing must avoid recursively routing the proxy outbound through the full tunnel. Burrow uses daemon-side outbound socket binding, analogous to Mihomo's default-interface outbound behavior, and installs resolved proxy-server excluded routes as a second guardrail. Selector changes remain daemon-local when they do not require platform route changes; if the selected node's resolved proxy-server routes change, the Network Extension should be restarted or have its settings reapplied so the excluded routes match the active outbound.
|
||||
|
||||
Selector changes should be daemon-local whenever socket binding is available. The Network Extension packet stream and route/DNS settings should stay up; the packet daemon updates its proxy outbound handle, and future flows read the new selection from that shared handle. This mirrors Mihomo's selector model more closely than restarting TUN for every node change.
|
||||
|
||||
Daemon-owned DNS/fake-IP metadata is part of the proxy packet runtime. The proxy-server route exclusion protects the selected outbound itself; ordinary website DNS is answered by the daemon and converted into hostname metadata for future flows. Mihomo-style rule selection can build on the same mapping without depending on ambient system DNS after the tunnel is active.
|
||||
|
||||
### Rollout Stages
|
||||
|
||||
1. Proposal and model cleanup.
|
||||
- Supersede `BEP-0009`.
|
||||
- Remove SOCKS runtime from the target design.
|
||||
2. Parser and preview.
|
||||
- Implement daemon-owned import preview for Mihomo-compatible YAML and URI-list inputs.
|
||||
- Add fixtures for Trojan and Shadowsocks.
|
||||
3. Storage and refresh.
|
||||
- Store proxy subscriptions as networks.
|
||||
- Preserve selected node and local ordering across refreshes.
|
||||
4. Packet engine foundation.
|
||||
- Add daemon packet-stream support for proxy networks.
|
||||
- Reconstruct TCP flows first with controlled DNS behavior.
|
||||
5. Protocol outbounds.
|
||||
- Add Trojan outbound.
|
||||
- Add Shadowsocks outbound.
|
||||
- Add UDP once NAT, timeout, and protocol support are explicit.
|
||||
6. Platform UI.
|
||||
- Add `Import Proxy Subscription` to Apple SwiftUI and Linux GTK add flows.
|
||||
- Add preview, warning, selection, refresh, and node status views to Apple SwiftUI, Linux GTK, and any future first-party UI surface.
|
||||
7. Compatibility and operations.
|
||||
- Add Mihomo comparison fixtures and sanitized live-check scripts.
|
||||
- Document remaining incompatibilities.
|
||||
|
||||
## Security and Operational Considerations
|
||||
|
||||
- Subscription URLs frequently contain bearer tokens. Full URLs must not appear in logs, screenshots, fixtures, crash reports, or generated docs.
|
||||
- Proxy node credentials must be redacted in all errors and diagnostics.
|
||||
- Insecure TLS settings imported from subscriptions must be visible in preview and stored as explicit risk metadata.
|
||||
- Refresh must have timeout, redirect, response-size, and content-type limits.
|
||||
- Parser failures must be non-fatal per node; malformed input should never panic the daemon.
|
||||
- Packet-tunnel activation must have a kill-switch path: when the tunnel stops or a proxy network is disabled, packet handling, DNS state, NAT state, and outbound sockets must be torn down.
|
||||
- If DNS mapping is fake-IP based, stale mappings must expire and must not leak between networks in ways that route traffic through the wrong node.
|
||||
- Full-tunnel mode must avoid recursive routing of the proxy server connection through the tunnel. On Apple, preferred bypass behavior is outbound socket binding to a non-tunnel interface. Route exclusions are fallback behavior for platforms or environments where socket binding is not available.
|
||||
- Removing SOCKS mode simplifies the threat model by leaving one supported runtime behavior for proxy networks.
|
||||
|
||||
## Contributor Playbook
|
||||
|
||||
1. Keep all UI subscription operations behind daemon gRPC.
|
||||
2. Update or remove current Trojan SOCKS runtime code instead of layering the packet-tunnel runtime beside it.
|
||||
3. Add parser fixtures for:
|
||||
- Clash/Mihomo YAML with `proxies`
|
||||
- proxy-provider content
|
||||
- base64 URI list
|
||||
- plaintext URI list
|
||||
- Trojan TCP, ws, grpc metadata
|
||||
- Shadowsocks legacy and SIP002-style links
|
||||
- unsupported protocols and malformed nodes
|
||||
- Mihomo parity cases copied from the behavior of `common/convert/converter.go`, especially Trojan `allowInsecure`, `sni`, `alpn`, `type=ws`, `type=grpc`, `fp`, and `pcs`
|
||||
4. Add packet-engine tests with synthetic TCP and DNS packets before full platform UI integration.
|
||||
5. Add a migration or cleanup path for any existing stored `NetworkType::Trojan` payloads created by the SOCKS-era implementation.
|
||||
6. Validate metadata and affected builds:
|
||||
|
||||
```bash
|
||||
uv run python Scripts/check-bep-metadata.py
|
||||
cargo test -p burrow proxy_subscription
|
||||
cargo check -p burrow
|
||||
```
|
||||
|
||||
7. For platform UI work, regenerate bindings after proto changes and verify Apple SwiftUI and Linux GTK still talk only to daemon gRPC for subscription operations.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- Keep local SOCKS mode. Rejected because it is not Burrow's VPN behavior, adds another support surface, and makes imported proxy subscriptions feel unlike WireGuard and Tailnet.
|
||||
- Use system `HTTP_PROXY` or SOCKS settings. Rejected because only proxy-aware apps participate, DNS behavior is inconsistent, and cleanup/rollback becomes OS-setting specific.
|
||||
- Shell out to Mihomo. Rejected as the default implementation because Burrow would inherit a large external runtime and a separate control plane. Mihomo remains the behavioral reference for import and TUN semantics.
|
||||
- Claim full Mihomo compatibility immediately. Rejected because Burrow should first support Trojan and Shadowsocks imports plus packet-tunnel routing correctly.
|
||||
- Treat every proxy node as a separate account. Rejected because nodes are endpoints under one imported network source, not identities.
|
||||
|
||||
## Impact on Other Work
|
||||
|
||||
- Supersedes `BEP-0009`.
|
||||
- Depends on the daemon IPC boundary in `BEP-0005`.
|
||||
- Aligns with the route and policy direction in `BEP-0003`.
|
||||
- May require a follow-up secret-storage BEP if existing network payload storage remains insufficient for long-lived proxy credentials.
|
||||
- Creates shared import and packet-engine foundations for future proxy protocols, but this proposal only covers Trojan and Shadowsocks.
|
||||
|
||||
## Decision
|
||||
|
||||
Pending.
|
||||
|
||||
## References
|
||||
|
||||
- Burrow daemon IPC boundary: `evolution/proposals/BEP-0005-daemon-ipc-and-apple-boundary.md`
|
||||
- Burrow route and negotiation roadmap: `evolution/proposals/BEP-0003-connect-ip-and-negotiation-roadmap.md`
|
||||
- Superseded Trojan SOCKS proposal: `evolution/proposals/BEP-0009-trojan-proxy-and-subscription-import.md`
|
||||
- Burrow tunnel schema: `proto/burrow.proto`
|
||||
- Burrow Apple tunnel provider: `Apple/NetworkExtension/PacketTunnelProvider.swift`
|
||||
- Burrow Apple UI: `Apple/UI/`
|
||||
- Burrow GTK UI: `burrow-gtk/src/`
|
||||
- Burrow current Trojan parser/runtime: `burrow/src/trojan.rs`
|
||||
- Burrow daemon runtime: `burrow/src/daemon/runtime.rs`
|
||||
- Mihomo config model: `config/config.go`
|
||||
- Mihomo proxy provider parsing: `adapter/provider/parser.go`
|
||||
- Mihomo provider content parsing: `adapter/provider/provider.go`
|
||||
- Mihomo URI conversion: `common/convert/converter.go`
|
||||
- Mihomo base64 fallback: `common/convert/base64.go`
|
||||
- Mihomo TUN listener lifecycle: `listener/listener.go`
|
||||
- Mihomo TUN stack setup: `listener/sing_tun/server.go`
|
||||
- Mihomo DNS hijack: `listener/sing_tun/dns.go`
|
||||
- Mihomo listener metadata bridge: `listener/sing/sing.go`
|
||||
- Mihomo TCP/UDP tunnel handling: `tunnel/tunnel.go`
|
||||
- Mihomo outbound parser: `adapter/parser.go`
|
||||
- Mihomo selector group: `adapter/outboundgroup/selector.go`
|
||||
- Mihomo Trojan outbound: `adapter/outbound/trojan.go`
|
||||
- Mihomo Shadowsocks outbound: `adapter/outbound/shadowsocks.go`
|
||||
Loading…
Add table
Add a link
Reference in a new issue