diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f19acfb..e18b9e0 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -19,35 +19,49 @@ contract is described here. state of its own — it connects, fetches the desired set of servers, and reconciles its running children against it. -## Workspace layout +## Components -A Cargo workspace with two crates: +`edison-stdiod` is a single binary that is both the long-lived service and the +control CLI. Its responsibilities — described here by role, independent of how +the source happens to be arranged on disk — are: ``` -crates/ - edison-stdiod/ the daemon + CLI binary - src/ - main.rs entry point / arg dispatch - cli/ subcommands: login, install, server, status, logs - daemon.rs the run loop: connect, reconcile, supervise - tunnel.rs WebSocket transport + framing - http.rs thin HTTP client for the backend REST surface - proc.rs child-process spawning and stdout/stderr pumps - state.rs state.json (atomic writes, consumed by the tray UI) - config.rs config.toml (backend URL, device id, credentials) - env_store.rs per-server environment variable storage - paths.rs platform config/log/data path resolution - platform/ macOS / Linux / Windows service integration - tunnel-protocol/ generated Rust types for the wire protocol -schema/ - tunnel-protocol.json JSON Schema — single source of truth for the protocol -dev/ - spike/ throwaway v0 prototype that informed the design + control commands ┌──────────────────────────────┐ + (login · install · │ Supervisor │ + status · logs · ──── config ──▶│ connect → reconcile → │ + server …) │ supervise the run loop │ + └────────┬───────────┬─────────┘ + │ │ + ┌───────────▼──┐ ┌────▼───────────┐ + │ Tunnel │ │ Child │ + │ transport │ │ supervision │ + │ (WebSocket, │ │ (spawn + stdio │ + │ opaque frames)│ │ pumps) │ + └───────┬───────┘ └────────┬───────┘ + outbound WS │ │ stdio + ▼ ▼ + Edison backend local MCP servers ``` -The `tunnel-protocol` crate's Rust types are generated from -`schema/tunnel-protocol.json` (via `schemars`/`typify`). The JSON Schema is the -single source of truth so the daemon and its peer can be kept in lock-step. +- **Control surface** — the CLI subcommands a user runs to authenticate, + register the OS service, manage servers, and inspect state. They persist + configuration; they do not carry MCP traffic. +- **Supervisor** — the long-lived run loop: connect, fetch desired state, + reconcile running children against it, and supervise. +- **Tunnel transport** — the single outbound WebSocket and its framing. It is + MCP-agnostic: MCP frames are forwarded as opaque bytes (see + [MCP-agnostic by design](#mcp-agnostic-by-design)). +- **Child supervision** — spawning each desired server as a subprocess and + pumping its stdio to and from the tunnel. + +Cross-cutting concerns sit beneath all of the above: the thin HTTP client for +the backend's REST surface, on-disk config and state persistence, and the +platform-specific service integration (macOS / Linux / Windows). + +The wire-protocol Rust types are **generated from a JSON Schema** (via +`schemars`/`typify`). The schema is the single source of truth so the daemon and +its peer can be kept in lock-step; see +[`schema/tunnel-protocol.json`](./schema/tunnel-protocol.json). ## Tunnel mechanism: reverse RPC over WebSocket diff --git a/README.md b/README.md index fc8ca5a..879968e 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,36 @@ -# stdiod +

stdiod

-[![CI](https://github.com/Edison-Watch/stdiod/actions/workflows/ci.yml/badge.svg)](https://github.com/Edison-Watch/stdiod/actions/workflows/ci.yml) -[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](./LICENSE) +

+Bridge local stdio MCP servers to a remote backend over a single outbound WebSocket — no inbound ports, and your processes, files, and credentials never leave the machine. +

-**stdiod** is a small daemon that bridges local [stdio MCP servers](https://modelcontextprotocol.io/) to the Edison Watch backend over a single outbound WebSocket tunnel. +

+ How it works • + Install • + Quickstart • + Configuration • + Architecture • + Development • + Credits +

-It runs on a user's machine, dials out to the backend (no inbound ports), and lets the backend drive locally-spawned MCP server subprocesses — forwarding MCP frames in both directions. An AI client talking to the backend's gateway reaches these local servers as if they were hosted remotely, while the processes (and their filesystem/credentials) stay on the user's device. +

+ CI + Version + License: AGPL v3 + Repo size +

-``` -AI client ──▶ Edison backend gateway ──▶ WebSocket tunnel ──▶ stdiod ──▶ local MCP server subprocess - (outbound, from the device) -``` +--- + +**stdiod** is a small Rust daemon that bridges local [stdio MCP servers](https://modelcontextprotocol.io/) to the Edison Watch backend over one outbound WebSocket tunnel. It runs on a user's machine, dials out to the backend (no inbound ports), and lets the backend drive locally-spawned MCP server subprocesses — forwarding MCP frames in both directions. An AI client talking to the backend's gateway reaches these local servers as if they were hosted remotely, while the processes (and their filesystem and credentials) stay on the user's device. -> **Status: experimental (v0.0.1).** This is early software under active development. It has **not** had an independent security audit. The wire protocol, CLI surface, and on-disk formats may change without notice before a 1.0 release. Today the daemon runs as a supervised service on **macOS only**; Linux and Windows support is on the roadmap (the CLI will tell you when a step is unsupported on your platform). +

+ An AI client reaches the Edison backend gateway, which drives the stdiod daemon over a single outbound WebSocket tunnel; the daemon spawns and supervises local stdio MCP servers on the user's machine. +

+ +> [!WARNING] +> **Experimental (v0.0.1).** Early software under active development; expect bugs. It has **not** had an independent security audit. The wire protocol, CLI surface, and on-disk formats may change without notice before a 1.0 release. Today the daemon runs as a supervised service on **macOS only** — Linux and Windows support is on the roadmap, and the CLI will tell you when a step is unsupported on your platform. ## How it works @@ -21,21 +39,28 @@ AI client ──▶ Edison backend gateway ──▶ WebSocket tunnel ── - **Child supervision.** The backend pushes a desired set of servers; the daemon spawns/stops the matching subprocesses and pumps their stdio. - **Survival.** It reconnects with backoff across network blips and machine sleep/resume, and reconciles desired state on every (re)connect. -See [`ARCHITECTURE.md`](./ARCHITECTURE.md) for the full design and [`schema/tunnel-protocol.json`](./schema/tunnel-protocol.json) for the wire protocol (the single source of truth for the frame types). +See [`ARCHITECTURE.md`](./ARCHITECTURE.md) for the full design and [`schema/tunnel-protocol.json`](./schema/tunnel-protocol.json) for the wire protocol — the single source of truth for the frame types. ## Install -Requires a [Rust toolchain](https://rustup.rs/) (the pinned channel is in [`rust-toolchain.toml`](./rust-toolchain.toml)). +Requires a [Rust toolchain](https://rustup.rs/) (the pinned channel is in [`rust-toolchain.toml`](./rust-toolchain.toml)). Build and install the `edison-stdiod` binary straight from a checkout: ```sh -# Build and install the `edison-stdiod` binary from a checkout: cargo install --path crates/edison-stdiod +``` + +The repository is a Cargo workspace; the `edison-stdiod` binary is the daemon **and** the control CLI. + +
+⚙️ Building in place (without installing) -# …or build in place: +```sh +git clone https://github.com/Edison-Watch/stdiod.git +cd stdiod cargo build --release # binary at target/release/edison-stdiod ``` -The repository is a Cargo workspace; the `edison-stdiod` binary is the daemon **and** the control CLI. +
## Quickstart @@ -60,7 +85,7 @@ To run the daemon in the foreground without installing a service unit (useful fo ```sh edison-stdiod run --backend http://localhost:3001 --api-key -# or rely on the persisted config from `login`: +# …or rely on the persisted config from `login`: edison-stdiod run ``` @@ -77,13 +102,48 @@ edison-stdiod server list edison-stdiod server remove filesystem ``` +## CLI Commands + +TLDR: `edison-stdiod --help` (and `edison-stdiod --help` for any subcommand). + +
+Expand + +| Command | What it does | +| --- | --- | +| `login` | Persist credentials + backend URL to `~/.config/edison-stdiod/config.toml` (mode `0600`). Merges on re-run, so you can rotate the API key without re-supplying the backend URL. | +| `install` | Register the OS supervisor unit (macOS LaunchAgent) so the daemon starts at login and restarts on crash. Requires `login` first. | +| `uninstall` | Stop and remove the supervisor unit. Pass `--purge` to also delete the persisted config and logs. | +| `run` | Run the daemon in the foreground (normally invoked by the service unit). Reads config or accepts `--backend` / `--api-key` / `--device-id` / `--label` flags (also via `EDISON_*` env vars). | +| `status` | Print a one-shot summary of supervisor-unit status, connection state, and currently-running child servers. | +| `logs` | Print the daemon log. `-f`/`--follow` to tail in real time; `-n`/`--lines N` to set the backscroll (default 200). | +| `server add ` | Register a stdio_tunnel server. `--command `, repeatable `--arg `, optional `--working-dir` and `--display-name`. The prefix name must be alphanumeric (plus hyphens). | +| `server list` | List stdio_tunnel servers registered for this device. `--json` for raw output. | +| `server remove ` | Delete a server by name. Idempotent — a missing name is reported as a no-op. | + +
+ ## Configuration +TLDR: `edison-stdiod login` writes everything to `~/.config/edison-stdiod/config.toml` (mode `0600`). + +
+Expand + Settings resolve in two layers, highest precedence first: 1. **CLI flags / environment variables** — handy for development overrides. 2. **`~/.config/edison-stdiod/config.toml`** — written by `edison-stdiod login`; this is what the OS supervisor unit reads (service units don't carry secrets in their environment). +```toml +# ~/.config/edison-stdiod/config.toml (mode 0600) +backend_url = "https://dashboard.edison.watch" # Backend base URL (http://localhost:3001 for dev) +api_key = "ew_live_…" # Bearer API key issued by the backend (plaintext, 0600) +edison_secret_key = "…" # Optional X-Edison-Secret-Key for per-user secret decryption +device_id = "my-laptop" # Stable device identifier; defaults to the machine hostname +device_label = "My Laptop" # Human-readable label shown in the dashboard +``` + | Field (`config.toml`) | Env var | Description | | --- | --- | --- | | `backend_url` | `EDISON_BACKEND_URL` | Backend base URL (`http://localhost:3001` for dev, `https://dashboard.edison.watch` for prod). | @@ -92,10 +152,70 @@ Settings resolve in two layers, highest precedence first: | `device_id` | `EDISON_DEVICE_ID` | Stable device identifier; defaults to the machine hostname. | | `device_label` | `EDISON_DEVICE_LABEL` | Human-readable label shown in the dashboard. | -Credentials live in a `0600` file under `~/.config/edison-stdiod/`. Rotate the API key by re-running `edison-stdiod login --api-key …`. To remove everything, run `edison-stdiod uninstall --purge`. +Rotate the API key by re-running `edison-stdiod login --api-key …`. To remove everything, run `edison-stdiod uninstall --purge`. + +
+ +## Files on disk + +TLDR: the daemon keeps almost nothing durable — the backend is the source of truth. + +
+Expand + +``` +~/.config/edison-stdiod/ + config.toml # backend URL, device_id, api_key, secret (mode 0600) + state.json # atomic writes; snapshot consumed by the desktop tray UI +~/Library/Logs/edison-stdiod/ # macOS — platform-equivalent paths elsewhere + daemon.log # rotated daily + child-.log # per-child stdout/stderr capture +``` + +The supervisor unit lives at `~/Library/LaunchAgents/watch.edison.stdiod.plist` on macOS (`KeepAlive=true`, `RunAtLoad=true`, no admin privileges needed). See [`ARCHITECTURE.md`](./ARCHITECTURE.md#persistence-and-survival) for Linux/Windows equivalents and the `state.json` schema. + +
+ +## Architecture + +TLDR: one outbound WebSocket carries a symmetric, MCP-agnostic frame protocol; the backend is the source of truth and the daemon reconciles local children against it. Full design in [`ARCHITECTURE.md`](./ARCHITECTURE.md). + +
+Expand + +``` + user's machine + ┌───────────────────────────────────────────────────────────┐ + │ │ + │ ┌──────────────┐ spawn / stdio ┌────────────────────┐│ + │ │ edison-stdiod │◀────────────────▶│ stdio MCP server(s) ││ + │ │ (daemon) │ pumps │ (child processes) ││ + │ └──────┬───────┘ └────────────────────┘│ + │ │ │ + └──────────┼──────────────────────────────────────────────────┘ + │ one outbound WebSocket (TLS:443, Bearer auth) + │ ▲ client_hello / device_status / announce_server + │ ▼ server_hello / desired_state_update / mcp_frame + ▼ + ┌────────────────────────┐ ┌──────────────┐ + │ Edison backend gateway │◀──────▶│ AI client │ + │ (source of truth) │ MCP │ │ + └────────────────────────┘ └──────────────┘ +``` + +- **Outbound-only & reverse RPC.** The daemon dials out; the backend drives it. Server-initiated frames (desired-state pushes, sampling requests, credential invalidations) are natural over the single long-lived connection. +- **MCP-agnostic transport.** The `tunnel` module treats each `frame` field as opaque bytes — MCP version bumps and new methods need no daemon changes. +- **Reconcile on (re)connect.** `client_hello` → `server_hello` (full desired-state snapshot) → diff and start/stop/restart children; steady-state changes arrive as `desired_state_update` deltas. + +
## Development +TLDR: `cargo build --workspace` then `cargo test --workspace`. + +
+Expand + ```sh cargo build --workspace # build cargo test --workspace # run tests @@ -103,10 +223,33 @@ cargo fmt --all --check # formatting cargo clippy --workspace --all-targets -- -D warnings # lints ``` +The `tunnel-protocol` crate's Rust types are generated from [`schema/tunnel-protocol.json`](./schema/tunnel-protocol.json) — keep the schema and the generated types in lock-step. + [`dev/spike/`](./dev/spike/) holds a throwaway v0 Python prototype that validated the wire protocol before the Rust daemon was written; it is kept as a historical record and is not part of the build. See [`CONTRIBUTING.md`](./CONTRIBUTING.md) for the contribution workflow and [`SECURITY.md`](./SECURITY.md) for how to report vulnerabilities. +
+ +## Credits + +This software is built with: + +- [Tokio](https://tokio.rs/) — async runtime +- [tokio-tungstenite](https://github.com/snapview/tokio-tungstenite) — WebSocket transport +- [reqwest](https://github.com/seanmonstar/reqwest) — HTTP client for the backend REST surface +- [clap](https://github.com/clap-rs/clap) — CLI parsing +- [serde](https://serde.rs/) + [serde_json](https://github.com/serde-rs/json) — serialization +- [tracing](https://github.com/tokio-rs/tracing) — structured logging + ## License Licensed under the [GNU Affero General Public License v3.0](./LICENSE). + +## Contributors + +
+ + + +Made with [contrib.rocks](https://contrib.rocks). diff --git a/docs/architecture.svg b/docs/architecture.svg new file mode 100644 index 0000000..708275c --- /dev/null +++ b/docs/architecture.svg @@ -0,0 +1,91 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + remote LLM API + third-party · optional + + + + + Edison backend + gateway + source of truth + + + + + + + + + USER'S MACHINE + + + + + AI client + local agent + + + + + stdiod + daemon + CLI + supervises children + + + + + + local stdio + MCP servers + child processes + + + + + + + + MCP + + + + ONE WEBSOCKET TUNNEL + outbound · TLS 443 · Bearer auth + no inbound ports — daemon dials out + + + + stdio + + + Processes, files, and credentials stay on the user's device — the daemon only dials out. +