diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8280b8b..be85dc5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -83,6 +83,7 @@ jobs: - run: pnpm db:verify-clean - run: pnpm lint - run: pnpm typecheck + - run: pnpm test:homelab-installer - run: pnpm test:unit - run: pnpm contracts:generate - name: Verify committed migrations diff --git a/README.md b/README.md index 3c6bdae..227752e 100644 --- a/README.md +++ b/README.md @@ -1,139 +1,306 @@

- Muster shield and tree logo + Muster shield and tree logo

Muster

-> Muster is the shared workspace for human and agent-driven security operations. +> The governed workspace for human and agent-assisted security operations. + +Muster brings alerts, investigations, approvals, evidence, task work, and +connector activity into durable security rooms. PostgreSQL is its authoritative +record; agents and queues execute bounded work around that record. Muster is a +coordination layer, not a replacement SIEM, EDR, SOAR, or case-management +product. + +![Synthetic Muster workspace showing a SOC room](docs/images/muster-security-workspace.png) + +The screenshot is a committed synthetic demo asset. It contains fictional +people, cases, and indicators only; do not treat it as a production capture. + +## Where Muster fits + +Muster intentionally leaves each adjacent product authoritative for its own +domain: + +- [Kelpie](https://github.com/jusso-dev/Kelpie) owns formal incident cases and + case lifecycle. Muster links case identifiers and posts governed case work; + it does not become the case system of record. +- [Tawny](https://github.com/jusso-dev/tawny) owns endpoint telemetry, + detections, bounded hunts, and approved endpoint response. Muster stores the + request, approval, delivery, and evidence references around that work. +- [Bower](https://github.com/jusso-dev/bower) owns application and legacy + telemetry delivery health. Muster presents its signals in operational rooms. + +## Current, verified surface + +The current application has room timelines, threads, reactions, mentions, +durable tasks, investigations, approvals, evidence upload, search, SSE updates, +agent readiness/activity, and responsive PWA navigation. The browser suite +exercises the room lifecycle, task delegation, approvals, agent activity, +governed connectors, clean install, accessibility, and mobile layouts; see +[`tests/`](tests). The health endpoint is +[`/api/v1/health`](apps/web/app/api/v1/health/route.ts). + +### Task board and starter missions + +[`/tasks`](apps/web/app/tasks/page.tsx) reads only organisation-scoped tasks. +It presents task status, assignment, run state, events, and governed delegation; +delegation carries an idempotency key and its result remains attached to that +task. It is not an autonomous-work queue: users with the applicable server-side +capabilities create, assign, delegate, review, or cancel bounded work. Any +connector-side action still needs its own capability and approval record. + +Starter agent definitions are installed with a clean database. Their work is +bounded by an explicit task, approved schedule, or configured watchlist and +remains reviewable through the [`/tasks` board](apps/web/app/tasks/page.tsx); +no agent is an unbounded background operator: + +- **Alfie** — evidence-backed threat and technology research. Administrators + create bounded, allowlisted watchlists at + [`/settings/alfie-research`](apps/web/app/settings/alfie-research/page.tsx); see the + [research runbook](docs/operations/alfie-research.md). It reads allowlisted + HTTPS feeds only, produces source-backed briefs, and does not perform external + actions. A starter mission: summarise a named technology from an approved + watchlist, then review its citations and feedback state. +- **Jessie** — bounded threat hunts and proposed Kelpie enrichment. It refuses + to run without a configured governed source; broader time, record, or + multi-source plans wait for human approval. A starter mission: investigate a + specific alert question, review recorded evidence and gaps, then decide + whether to approve the exact hunt or proposed enrichment. +- **Parker** — operational reporting with organisation-scoped weekly or + monthly schedules at [`/settings/parker-reports`](apps/web/app/settings/parker-reports/page.tsx). + A due occurrence creates one Parker-assigned review task; delegating that task + produces a reproducible report manifest, review/version record, and room post. + Email delivery remains separately approved. See the + [Parker report runbook](docs/operations/parker-reports.md). A starter mission: + prepare an analyst, leadership, or executive report for a selected room and + period, then review the manifest before approving delivery. + +For a disposable evaluation, create three explicit tasks rather than treating +the agents as autonomous staff: ask Alfie to summarise an allowlisted research +topic with citations, Jessie to perform a bounded hunt against a linked alert, +and Parker to prepare a leadership report for a selected room and period. +Review the resulting evidence, run record, room post, and any approval before +acting. The committed demo data is synthetic and separate from a clean install. + +All external actions remain server-side capability checked and approval gated. +Codex receives read-only, no-network execution with typed output validation; +it cannot directly perform connector writes. + +### Mock and real connector status + +Both supplied Compose topologies set `MUSTER_MOCK_INTEGRATIONS=true` and start +synthetic Kelpie, Tawny, and Bower services. Their replies are demo/test data; +a mock health check, query, or action result is never production delivery. + +| Product | Supplied Compose default | Configured endpoint surface | Real status | +| ------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------ | +| Kelpie | Synthetic mock | [`/integrations/connectors`](apps/web/app/integrations/connectors/page.tsx) has governed Kelpie query/case-management configuration. | Code and contract tests exist; no live-product certification is claimed. | +| Tawny | Synthetic mock | Connector administration supports read-only Tawny and separately governed Tawny-response configuration. | Code and contract tests exist; no live-product certification is claimed. | +| Bower | Synthetic mock | No Bower option exists in connector administration; product pages are demo-only. | Mock/demo only; do not treat it as a configured production connector. | + +Configured Kelpie and Tawny credentials are stored server-side per organisation; +each endpoint, capability, and approval path must be validated in the target +environment. See [current upstream contracts](docs/integrations/current-upstream-contracts.md). + +## Clean Docker quick start + +Supported host: Intel/AMD64 Ubuntu with Docker Engine and the Compose plugin. +The clean installer creates a fresh database, object store, local administrator, +and agent definitions; it does **not** seed synthetic alerts, cases, messages, +or tasks. -Muster connects application telemetry, endpoint detections, security investigations and incident case management in one auditable workspace. - -**Bring the signal together.** +```bash +git clone https://github.com/jusso-dev/Muster.git +cd Muster +./scripts/bootstrap.sh +``` -![Muster security operations workspace](docs/images/muster-security-workspace.png) +Open `http://localhost:3000`. The script writes `.env` locally and prints the +generated administrator password. The value is local `.env` state, not a demo +credential: do not commit it and change it for any non-disposable installation. +Local Compose exposes PostgreSQL, Redis, MinIO, Mailpit, the web application, +and synthetic connector mocks for development; it is not a hardened +Internet-facing topology. -Muster is a self-hosted workspace where analysts, responders, engineers, security products, and permission-scoped agents work in persistent security rooms. Signals, investigations, approvals, response actions, and linked cases arrive as channel activity instead of separate operational dashboards. Muster complements—not replaces—SIEM, EDR, SOAR, or formal case-management systems. +For source-based development instead: -## Connected security stack +```bash +pnpm install --frozen-lockfile +pnpm db:migrate +pnpm db:bootstrap +pnpm dev +``` -Muster is most useful alongside: +### Demo and screenshots are separate -- [Kelpie](https://github.com/jusso-dev/Kelpie) for authoritative incident response and formal security case management -- [Tawny](https://github.com/jusso-dev/tawny) for endpoint telemetry, detections, hunting, and bounded response actions -- [Bower](https://github.com/jusso-dev/bower) for trusted application and legacy-system telemetry delivery health +Only populate a disposable database when explicitly preparing tests or +screenshots: -Muster links their signals and actions into rooms without duplicating their -authoritative data models. +```bash +MUSTER_DEMO_MODE=true NEXT_PUBLIC_MUSTER_DEMO_MODE=true pnpm db:seed +``` -## What works +Never run that command against a production or clean-install database. -- Slack-familiar security rooms with channels, direct messages, threads, reactions, mentions, structured event cards, drafts, SSE updates, and responsive navigation -- A durable task board for assigning bounded work to analysts or permission-scoped Codex agents, with human review and approval gates for external actions -- Alert, investigation, approval, response, evidence, and linked-case activity rendered directly into durable room timelines -- Kelpie case, Tawny endpoint, and Bower telemetry-health adapters with explicit mock mode -- Versioned MSEP contracts, signed ingestion, replay protection, JSON Schema generation, and typed client primitives -- PostgreSQL-scoped domain services, transactional outbox, nine policy-separated BullMQ queues, idempotency, and hash-chained audit events -- Better Auth password, verification, TOTP, recovery-code, passkey, OIDC/Entra-ready configuration -- Capability-based authorisation and default approval policy for response actions -- Subscription-backed Codex agent runtime, typed outputs, read-only/no-network isolation, cancellation, kill switch, and governed continuous learning -- PWA shell with safe offline page and local draft preservation; sensitive data is not cached offline +| Mode | Data and route behaviour | +| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Clean install | `bootstrap.sh` creates local configuration, an administrator, starter rooms, and starter agent definitions. It does not seed synthetic alerts, cases, messages, tasks, or a fixed demo login. | +| Demo/screenshot | `MUSTER_DEMO_MODE=true` plus `pnpm db:seed` adds synthetic data. Product integration and workflow views are demo-only and redirect in clean mode. | -## Quick start +## Homelab image install (port 3004) -Requirements: Docker Compose, or Node.js 24+, pnpm 11, PostgreSQL 17+, Redis 8+, and S3-compatible storage. +The public image is built for `linux/amd64`; CI publishes SBOM/provenance and +verifies an anonymous GHCR pull on `main`. Use a reviewed OCI digest, not +`latest`. At this README revision, the published `sha-a37ea88` image resolves to +`ghcr.io/jusso-dev/muster@sha256:75ebdad962373ff1fa5dbef8dba8f0a005de6058e21655dad8c72b1129e90861`. +Verify that digest or replace it with a newer reviewed release before deployment. ```bash -./scripts/bootstrap.sh +git clone https://github.com/jusso-dev/Muster.git +cd Muster +MUSTER_PUBLIC_URL=http://muster.example.lan:3004 \ +AUTH_TRUSTED_ORIGINS=http://muster.example.lan:3004 \ +MUSTER_IMAGE=ghcr.io/jusso-dev/muster@sha256:75ebdad962373ff1fa5dbef8dba8f0a005de6058e21655dad8c72b1129e90861 \ +./scripts/install-homelab.sh ``` -Or start everything directly: +`install-homelab.sh` persists an explicitly supplied `MUSTER_IMAGE` or legacy +`MUSTER_VERSION` in `.env.homelab`; inspect that file before later upgrades. +For a newer release, change the persisted reference only after its image, +SBOM/provenance, migrations, and rollback path have been reviewed. -```bash -docker compose up --build -``` +The homelab Compose file publishes only web port `3004` by default and keeps +PostgreSQL, Redis, MinIO, Mailpit, worker, gateway, and mocks internal. Set +`MUSTER_PUBLIC_URL` and `AUTH_TRUSTED_ORIGINS` to the exact browser origin(s) +that will use the service. When using HTTPS behind a reverse proxy, set +`AUTH_SECURE_COOKIES=true`; do not add broad wildcard origins. -Published releases are available from GitHub Container Registry: - -```bash -docker pull ghcr.io/jusso-dev/muster:latest -``` +The installer creates `.env.homelab` with mode `600`. Keep it out of source +control. The generated topology still uses synthetic connector endpoints until +you deliberately configure governed real connectors. -The default CI workflow publishes an Intel/AMD `linux/amd64` image as `latest`, -version tags, and immutable SHA tags with SBOM and provenance on pushes to -`main`. Its final publication gate logs out of GHCR and verifies an anonymous -pull, so CI fails if the package is not public. +### Codex subscription authentication -Muster uses the Codex SDK and your ChatGPT Codex subscription for agent runs. -Authenticate the persistent private Docker volume once: +Agents use a ChatGPT Codex subscription, not an OpenAI API key. Authenticate +the private persistent volume interactively after the stack is running: ```bash docker compose --profile setup run --rm codex-login ``` -The gateway never uses the credential as an OpenAI API key. Codex runs receive -organisation-scoped PostgreSQL context, typed output schemas, an empty read-only -workspace, disabled network/web search, and no action approval. State-changing -security actions still use Muster tools and approval records outside Codex. - -For a single-node homelab installation that pulls the public image: +For the homelab topology, use its Compose file and generated environment file: ```bash -./scripts/install-homelab.sh +docker compose --env-file .env.homelab \ + -f deploy/docker/docker-compose.homelab.yml \ + --profile setup run --rm codex-login ``` -The homelab example listens on port `3004` and trusts both -`http://192.168.1.19:3004` and `http://homelab:3004`. - -Open: - -- Muster: http://localhost:3000 -- Mailpit: http://localhost:8025 -- MinIO console: http://localhost:9001 +The normal Compose gateway and setup job share the private `muster-codex` +volume; the homelab topology calls the same private state volume `codex-state`. +Do not mount either volume into the web service, copy its contents into a +repository, or print it in logs. If no authorised Codex session exists, agent +readiness correctly reports that limitation rather than pretending a run worked. -The bootstrap prints generated local credentials. New installations contain only -the workspace administrator, empty starter rooms, and permission-scoped agent -definitions—no synthetic operational activity. Demonstration data is opt-in and -reserved for tests and screenshot generation. Mock integrations remain visibly -labelled; mock success is never represented as production delivery. - -## Architecture +## Operating model ```mermaid flowchart LR - B[Browser/PWA] -->|HTTP commands| W[Next.js web] - W --> P[(PostgreSQL)] - W -->|transactional outbox| P - W -->|SSE| B - P --> O[Outbox dispatcher] - O --> Q[Redis + BullMQ] + B[Browser / PWA] --> W[Next.js web] + W --> P[(PostgreSQL authoritative record)] + W -->|transactional outbox| Q[Redis + BullMQ] Q --> K[Worker] Q --> G[Agent gateway] - K --> E[S3-compatible evidence] - K --> X[Kelpie · Tawny · Bower · Sentinel] - G --> R[Codex SDK · ChatGPT subscription] - K -->|ephemeral fan-out| W + K --> E[Private evidence storage] + K --> X[Governed Kelpie / Tawny / Bower connectors] + G --> C[Codex subscription runtime] ``` -PostgreSQL is authoritative. Redis holds execution and ephemeral fan-out state only. Kelpie remains authoritative for formal cases; Tawny for endpoint telemetry and bounded response; Bower for application telemetry selection and delivery evidence. - -See [architecture](docs/architecture/README.md), [current upstream contracts](docs/integrations/current-upstream-contracts.md), and [threat model](docs/security/threat-model.md). - -## Development +Redis and BullMQ are execution infrastructure, not a source of truth. Significant +state changes, audit events, and outbox records are written transactionally. +Incoming connector content is untrusted evidence, not agent instruction. + +Read the [architecture](docs/architecture/README.md), +[connector contract notes](docs/integrations/current-upstream-contracts.md), and +[OpenAPI description](docs/openapi.yaml) before extending integrations. The +[deployment guide](docs/operations/deployment.md) explains the supported +single-node/evaluation topology; [CONTRIBUTING.md](CONTRIBUTING.md) defines the +required design, test, migration, and rollback review for changes. + +## Security and operations + +- Every domain query is organisation scoped; routes, workers, and agent tools + perform server-side capability checks. +- Dangerous actions require approval records. Evidence uses private object + storage, hash verification, classification, quarantine metadata, and audit + history. +- Agent learning is evidence-backed, versioned, evaluated, approved, and + reversible; it cannot grant new permissions or tools. +- Review [SECURITY.md](SECURITY.md), the [threat model](docs/security/threat-model.md), + [authentication and capabilities](docs/security/authentication-and-capabilities.md), + and [agent safety](docs/security/agent-safety.md) before deployment. + +Back up PostgreSQL and the versioned evidence bucket; Redis is rebuildable. +The required restore order and verification checks are in +[backup and restore](docs/operations/backup-restore.md). For a suspected Muster +compromise, follow [incident recovery](docs/operations/incident-recovery.md). + +### Troubleshooting + +- **Web is unhealthy:** run `docker compose ps`, then inspect the failing + service logs. For homelab use `docker compose --env-file .env.homelab -f +deploy/docker/docker-compose.homelab.yml ps`. The web health route is + `/api/v1/health`; see the [deployment guide](docs/operations/deployment.md). +- **Login loops or callback errors:** verify `MUSTER_PUBLIC_URL`, + `AUTH_TRUSTED_ORIGINS`, reverse-proxy headers, and `AUTH_SECURE_COOKIES`; then + restart the web service. +- **Agent is not ready:** use the Agents view to distinguish missing Codex + authentication from disabled/readiness evidence. Re-run `codex-login` only + on the private volume; see [deployment guidance](docs/operations/deployment.md). +- **Connector delivery does not happen:** confirm it is not a mock, then check + the connector health, organisation capability, approval record, outbox, and + upstream product logs. Do not replay raw queue jobs. + +## Known limits and roadmap + +Muster is an MVP reference implementation. It does not make the local Compose +topology production-ready: plan identity policy, TLS/reverse proxy, egress +controls, malware scanning, object lock/retention, backup restore exercises, +monitoring, and high availability for your environment. The default external +products are mocks. A first-class Slack surface and portable external agent +harnesses are upcoming work tracked in [#33](https://github.com/jusso-dev/Muster/issues/33): there is no Slack app/event endpoint or external harness contract to configure yet. + +## Testing and contribution checks + +For source tests, use Node 24+, pnpm 11.17.0, Docker, and installed Chromium +browser dependencies. The application, worker, and gateway test servers are +started by Playwright; PostgreSQL and Redis are the required local services. ```bash -pnpm install +pnpm install --frozen-lockfile +docker compose up -d postgres redis minio minio-init pnpm db:migrate pnpm db:bootstrap -pnpm dev +pnpm check +pnpm exec playwright install --with-deps chromium +pnpm test:e2e -- --project=chromium +pnpm exec playwright test --config=playwright.clean.config.ts ``` -To populate a disposable database for screenshots or tests only: +The clean-install suite proves that no synthetic demo data is required. The +standard suite uses synthetic mocks and test mode; it is not real-connector +certification. Run `pnpm screenshots` only against synthetic data. Before +submitting a change, follow [CONTRIBUTING.md](CONTRIBUTING.md), keep the +organisation/capability/approval boundaries intact, and include the relevant +unit, integration, browser, migration, and rollback evidence. -```bash -MUSTER_DEMO_MODE=true NEXT_PUBLIC_MUSTER_DEMO_MODE=true pnpm db:seed -``` +CI currently runs clean-install and synthetic-demo browser checks with Chromium +on Ubuntu `linux/amd64`. `pnpm test:e2e` also defines local Firefox, WebKit, and +iPhone 13-emulation projects; those projects are useful regression checks, not +a claim of a production-browser support or connector-certification matrix. -Quality gates: +## Additional verification commands ```bash pnpm lint @@ -144,23 +311,6 @@ pnpm test:e2e pnpm screenshots ``` -Public API contracts are under `/api/v1`; see [OpenAPI](docs/openapi.yaml). Generate MSEP JSON Schemas with `pnpm contracts:generate`. - -## Security model - -- Every domain record and query is organisation scoped. -- Routes, services, workers, integration tools, and agent tools enforce capabilities server-side. -- Dangerous state changes require an approval record; detection publication requires two approvers; evidence deletion is prohibited. -- Telemetry, files, URLs, documents, comments, and tool results enter prompts only as `untrusted_evidence`. -- Agent memories are evidence-backed. Skill proposals are immutable, evaluated, human-approved, versioned, and reversible; they cannot expand their own tools, permissions, data allowance, runtime, token, or cost limits. -- Evidence uses private object storage, short-lived access, hash verification, classification, quarantine, and audit metadata. - -Read [SECURITY.md](SECURITY.md) before production use. - -## Project status - -This repository is an MVP reference implementation. External-product mocks are suitable only for local demonstration. Validate real connector versions, identity policies, retention, object lock, malware scanning, egress controls, backups, and high-availability design before production rollout. - ## License Apache-2.0. See [LICENSE](LICENSE). diff --git a/deploy/docker/.env.homelab.example b/deploy/docker/.env.homelab.example index 0c7bfa3..684bdb4 100644 --- a/deploy/docker/.env.homelab.example +++ b/deploy/docker/.env.homelab.example @@ -1,6 +1,6 @@ -MUSTER_PUBLIC_URL=http://192.168.1.19:3004 +MUSTER_PUBLIC_URL=http://muster.example.lan:3004 MUSTER_HTTP_PORT=3004 -AUTH_TRUSTED_ORIGINS=http://192.168.1.19:3004,http://homelab:3004 +AUTH_TRUSTED_ORIGINS=http://muster.example.lan:3004,http://homelab:3004 MUSTER_ORGANISATION_NAME=Muster Workspace MUSTER_ORGANISATION_SLUG=muster MUSTER_DEFAULT_TIMEZONE=Australia/Sydney diff --git a/deploy/docker/README.md b/deploy/docker/README.md index e4e06bd..c80d260 100644 --- a/deploy/docker/README.md +++ b/deploy/docker/README.md @@ -6,12 +6,16 @@ The homelab profile pulls the public GHCR image and keeps PostgreSQL, Redis, MinIO, Mailpit, and synthetic product mocks on the internal Compose network: ```bash -./scripts/install-homelab.sh +MUSTER_IMAGE=ghcr.io/jusso-dev/muster@sha256:75ebdad962373ff1fa5dbef8dba8f0a005de6058e21655dad8c72b1129e90861 \ + ./scripts/install-homelab.sh ``` The installer creates a mode-600 `.env.homelab`, generates independent database, authentication, storage, and administrator secrets, pulls the public -image, starts the stack, waits for health, and creates the local administrator. -Only Muster's configured HTTP port is published. The default uses plain HTTP -for the trusted local network at `192.168.1.19`; use an HTTPS reverse proxy and set -`AUTH_SECURE_COOKIES=true` when exposing Muster beyond it. +reviewed digest, starts the stack, waits for health, and creates the local +administrator. Replace the digest only after reviewing the corresponding +release evidence; mutable tags such as `latest` are rejected. +Only Muster's configured HTTP port is published. The example uses +`http://muster.example.lan:3004`; replace it with the exact trusted browser +origin. Use an HTTPS reverse proxy and set `AUTH_SECURE_COOKIES=true` when +exposing Muster beyond a trusted local network. diff --git a/deploy/docker/docker-compose.homelab.yml b/deploy/docker/docker-compose.homelab.yml index da1de3b..6310d20 100644 --- a/deploy/docker/docker-compose.homelab.yml +++ b/deploy/docker/docker-compose.homelab.yml @@ -33,7 +33,7 @@ x-muster-environment: &muster-environment CODEX_HOME: /var/lib/muster/codex x-muster-service: &muster-service - image: ghcr.io/jusso-dev/muster:${MUSTER_VERSION:-latest} + image: ${MUSTER_IMAGE:-ghcr.io/jusso-dev/muster:${MUSTER_VERSION:-latest}} pull_policy: always restart: unless-stopped environment: *muster-environment diff --git a/docs/operations/deployment.md b/docs/operations/deployment.md index dc481c9..380d747 100644 --- a/docs/operations/deployment.md +++ b/docs/operations/deployment.md @@ -23,7 +23,8 @@ The Codex credential is runtime state, not image content. Authenticate once: docker compose --profile setup run --rm codex-login ``` -Compose stores the resulting `auth.json` in the private `codex-state` volume. +The root Compose topology stores `auth.json` in private `muster-codex` volume; +the homelab topology calls its corresponding private volume `codex-state`. Restrict Docker access, never publish or log this file, and revoke the Codex session if the host is compromised. The gateway reports `authentication_required` until the credential exists. diff --git a/package.json b/package.json index e7c30f5..2e6fbb9 100644 --- a/package.json +++ b/package.json @@ -17,6 +17,7 @@ "test:unit": "vitest run", "test:e2e": "playwright test", "test:e2e:homelab": "playwright test --config=playwright.homelab.config.ts", + "test:homelab-installer": "bash scripts/test-install-homelab.sh", "contracts:generate": "pnpm --filter @muster/contracts generate", "db:generate": "pnpm --filter @muster/database generate", "db:migrate": "pnpm --filter @muster/database migrate", diff --git a/scripts/install-homelab.sh b/scripts/install-homelab.sh index d112ec0..2d75e42 100755 --- a/scripts/install-homelab.sh +++ b/scripts/install-homelab.sh @@ -5,8 +5,31 @@ cd "$(dirname "$0")/.." env_file=".env.homelab" compose_file="deploy/docker/docker-compose.homelab.yml" -requested_public_url="${MUSTER_PUBLIC_URL:-http://192.168.1.19:3004}" +requested_public_url="${MUSTER_PUBLIC_URL:-http://muster.example.lan:3004}" requested_http_port="${MUSTER_HTTP_PORT:-3004}" +requested_version="${MUSTER_VERSION:-}" +requested_image="${MUSTER_IMAGE:-}" + +read_env_value() { + sed -n "s|^$1=||p" "$env_file" | tail -n 1 +} + +if [[ -n "$requested_version" && -n "$requested_image" ]]; then + printf 'Set either MUSTER_VERSION or MUSTER_IMAGE, not both.\n' >&2 + exit 2 +fi +if [[ ! -f "$env_file" && -z "$requested_version" && -z "$requested_image" ]]; then + printf 'A reviewed immutable MUSTER_IMAGE digest or MUSTER_VERSION sha tag is required for a fresh install.\n' >&2 + exit 2 +fi +if [[ -n "$requested_version" && ! "$requested_version" =~ ^sha-[0-9a-f]{7,40}$ ]]; then + printf 'MUSTER_VERSION must be an immutable sha- tag.\n' >&2 + exit 2 +fi +if [[ -n "$requested_image" && ! "$requested_image" =~ ^ghcr\.io/jusso-dev/muster@sha256:[0-9a-f]{64}$ ]]; then + printf 'MUSTER_IMAGE must be the reviewed ghcr.io/jusso-dev/muster@sha256: reference.\n' >&2 + exit 2 +fi if [[ ! -f "$env_file" ]]; then cp deploy/docker/.env.homelab.example "$env_file" @@ -39,9 +62,65 @@ if ! grep -q '^AUTH_TRUSTED_ORIGINS=' "$env_file"; then "$requested_public_url" "$requested_http_port" >> "$env_file" fi -set -a -source "$env_file" -set +a +# Persist an explicitly requested reviewed image reference before sourcing the env +# file. Otherwise the template's `latest` value would silently win. +if [[ -n "$requested_version" ]]; then + sed -i.bak \ + -e "s|^MUSTER_VERSION=.*|MUSTER_VERSION=${requested_version}|" \ + -e '/^MUSTER_IMAGE=/d' \ + "$env_file" + if ! grep -q '^MUSTER_VERSION=' "$env_file"; then + printf 'MUSTER_VERSION=%s\n' "$requested_version" >> "$env_file" + fi + rm "$env_file.bak" +fi + +if [[ -n "$requested_image" ]]; then + sed -i.bak \ + -e "s|^MUSTER_IMAGE=.*|MUSTER_IMAGE=${requested_image}|" \ + -e '/^MUSTER_VERSION=/d' \ + "$env_file" + if ! grep -q '^MUSTER_IMAGE=' "$env_file"; then + printf 'MUSTER_IMAGE=%s\n' "$requested_image" >> "$env_file" + fi + rm "$env_file.bak" +fi + +effective_version="$(read_env_value MUSTER_VERSION)" +effective_image="$(read_env_value MUSTER_IMAGE)" +if [[ -n "$effective_version" && -n "$effective_image" ]]; then + printf 'The persisted homelab configuration contains conflicting image references.\n' >&2 + exit 2 +fi +if [[ -n "$effective_version" ]]; then + if [[ ! "$effective_version" =~ ^sha-[0-9a-f]{7,40}$ ]]; then + printf 'The persisted MUSTER_VERSION is not an immutable sha- tag.\n' >&2 + exit 2 + fi +elif [[ -n "$effective_image" ]]; then + if [[ ! "$effective_image" =~ ^ghcr\.io/jusso-dev/muster@sha256:[0-9a-f]{64}$ ]]; then + printf 'The persisted MUSTER_IMAGE is not a reviewed GHCR digest.\n' >&2 + exit 2 + fi +else + printf 'The persisted homelab configuration has no immutable image reference.\n' >&2 + exit 2 +fi + +MUSTER_HTTP_PORT="$(read_env_value MUSTER_HTTP_PORT)" +MUSTER_PUBLIC_URL="$(read_env_value MUSTER_PUBLIC_URL)" +MUSTER_LOCAL_ADMIN_EMAIL="$(read_env_value MUSTER_LOCAL_ADMIN_EMAIL)" +MUSTER_LOCAL_ADMIN_PASSWORD="$(read_env_value MUSTER_LOCAL_ADMIN_PASSWORD)" +if [[ -z "$MUSTER_LOCAL_ADMIN_EMAIL" || -z "$MUSTER_LOCAL_ADMIN_PASSWORD" ]]; then + printf 'Homelab administrator credentials are missing from %s.\n' "$env_file" >&2 + exit 2 +fi + +for external_network in tawny_default kelpie_default; do + if ! docker network inspect "$external_network" >/dev/null 2>&1; then + docker network create "$external_network" >/dev/null + fi +done docker compose --env-file "$env_file" -f "$compose_file" pull docker compose --env-file "$env_file" -f "$compose_file" up -d diff --git a/scripts/test-install-homelab.sh b/scripts/test-install-homelab.sh new file mode 100755 index 0000000..82f6d48 --- /dev/null +++ b/scripts/test-install-homelab.sh @@ -0,0 +1,105 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root="$(cd "$(dirname "$0")/.." && pwd)" +fixture_root="$(mktemp -d)" +trap 'rm -rf "$fixture_root"' EXIT + +mkdir -p "$fixture_root/scripts" "$fixture_root/deploy/docker" "$fixture_root/bin" +cp "$repo_root/scripts/install-homelab.sh" "$fixture_root/scripts/" +cp "$repo_root/deploy/docker/.env.homelab.example" "$fixture_root/deploy/docker/" +cp "$repo_root/deploy/docker/docker-compose.homelab.yml" "$fixture_root/deploy/docker/" + +cat >"$fixture_root/bin/openssl" <<'EOF' +#!/usr/bin/env bash +length="${*: -1}" +printf '%*s\n' "$((length * 2))" '' | tr ' ' a +EOF + +cat >"$fixture_root/bin/docker" <<'EOF' +#!/usr/bin/env bash +printf '%s\n' "$*" >>"${DOCKER_CALLS:?}" +if [[ "${1:-}" == "network" && "${2:-}" == "inspect" ]]; then + grep -Fxq "${3:-}" "${DOCKER_NETWORKS:?}" 2>/dev/null + exit +fi +if [[ "${1:-}" == "network" && "${2:-}" == "create" ]]; then + printf '%s\n' "${3:-}" >>"${DOCKER_NETWORKS:?}" +fi +EOF + +cat >"$fixture_root/bin/curl" <<'EOF' +#!/usr/bin/env bash +if [[ " $* " == *" --write-out "* ]]; then + printf '200' +fi +EOF + +cat >"$fixture_root/bin/sleep" <<'EOF' +#!/usr/bin/env bash +exit 0 +EOF + +chmod +x "$fixture_root/bin/"* +export PATH="$fixture_root/bin:$PATH" +export DOCKER_CALLS="$fixture_root/docker-calls" +export DOCKER_NETWORKS="$fixture_root/docker-networks" +touch "$DOCKER_CALLS" "$DOCKER_NETWORKS" + +if ( + cd "$fixture_root" + ./scripts/install-homelab.sh >/dev/null 2>&1 +); then + printf 'Fresh install accepted a missing immutable image reference.\n' >&2 + exit 1 +fi + +digest="ghcr.io/jusso-dev/muster@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +( + cd "$fixture_root" + MUSTER_PUBLIC_URL=http://muster.test:3004 \ + AUTH_TRUSTED_ORIGINS=http://muster.test:3004 \ + MUSTER_IMAGE="$digest" \ + ./scripts/install-homelab.sh >/dev/null +) +grep -Fxq "MUSTER_IMAGE=$digest" "$fixture_root/.env.homelab" +if grep -q '^MUSTER_VERSION=' "$fixture_root/.env.homelab"; then + printf 'Digest install retained conflicting MUSTER_VERSION.\n' >&2 + exit 1 +fi +grep -Fxq "tawny_default" "$DOCKER_NETWORKS" +grep -Fxq "kelpie_default" "$DOCKER_NETWORKS" + +( + cd "$fixture_root" + MUSTER_VERSION=sha-bbbbbbb ./scripts/install-homelab.sh >/dev/null +) +grep -Fxq "MUSTER_VERSION=sha-bbbbbbb" "$fixture_root/.env.homelab" +if grep -q '^MUSTER_IMAGE=' "$fixture_root/.env.homelab"; then + printf 'Tag transition retained conflicting MUSTER_IMAGE.\n' >&2 + exit 1 +fi + +second_digest="ghcr.io/jusso-dev/muster@sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc" +( + cd "$fixture_root" + MUSTER_IMAGE="$second_digest" ./scripts/install-homelab.sh >/dev/null +) +grep -Fxq "MUSTER_IMAGE=$second_digest" "$fixture_root/.env.homelab" +if grep -q '^MUSTER_VERSION=' "$fixture_root/.env.homelab"; then + printf 'Digest transition retained conflicting MUSTER_VERSION.\n' >&2 + exit 1 +fi + +sed -i.bak -e '/^MUSTER_IMAGE=/d' "$fixture_root/.env.homelab" +rm "$fixture_root/.env.homelab.bak" +printf 'MUSTER_VERSION=latest\n' >>"$fixture_root/.env.homelab" +if ( + cd "$fixture_root" + ./scripts/install-homelab.sh >/dev/null 2>&1 +); then + printf 'Installer accepted a persisted mutable image tag.\n' >&2 + exit 1 +fi + +printf 'Homelab installer transitions passed.\n'