From 6fc94527093103b49583a915932d1a322bdc9aff Mon Sep 17 00:00:00 2001 From: Maciek Date: Thu, 30 Jul 2026 10:15:27 +0200 Subject: [PATCH] docs(readme): fix recursor stack, versions, setup steps, architecture diagram, and feature list Signed-off-by: Maciek --- Makefile | 3 +- README.md | 125 +++++++++++++++++++++++++++++++++--------------------- 2 files changed, 77 insertions(+), 51 deletions(-) diff --git a/Makefile b/Makefile index 42ad41d3..0643c007 100644 --- a/Makefile +++ b/Makefile @@ -27,8 +27,7 @@ down: ## Stops all of the services. -f compose.dnscheck.yml \ -f compose.sdns.yml \ -f compose.knot.yml \ - down; \ - docker kill -a + down up_dns: ## Starts the DNS services (both recursors: sdns + knot). docker compose \ diff --git a/README.md b/README.md index fefb7e01..1f614202 100644 --- a/README.md +++ b/README.md @@ -20,71 +20,77 @@ modDNS is a full-stack DNS security platform that combines encrypted DNS transpo modDNS is built as a microservices architecture with the following components: ``` -┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ -│ │───▶│ │───▶│ │───▶│ │ -│ Web Client │ │ Nginx Proxy │ │ Frontend │ │ API Server │ -│ │ │ │ │ (React) │ │ │ -└──────────────┘ └──────────────┘ └──────────────┘ └──────┬───────┘ - │ - ┌──────────┴──────────┐ - │ │ - ▼ ▼ - ┌──────────────┐ ┌──────────────┐ - │ │ │ │ - │ Redis │ │ MongoDB │ - │ (Caching) │ │ (Storage) │ - └──────────────┘ └──────────────┘ - ▲ - │ -┌──────────────┐ ┌──────────────┐ │ -│ │───▶│ │────────────────────┘ -│ DNS Clients │ │ DNS Proxy │ -│ │ │ │ -└──────────────┘ └──────┬───────┘ - │ - ▼ - ┌──────────────┐ - │ │ - │ DNS Resolver │ - │(SDNS/Unbound)│ - └──────────────┘ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Web Client │──▶│ Nginx │──▶│ Frontend │ +│ (browser) │ │ (:80, static)│ │ (React SPA) │ +└──────┬───────┘ └──────────────┘ └──────────────┘ + │ REST API (:3000) + ▼ +┌──────────────┐ ┌──────────────────────┐ +│ API Server │──▶│ Redis │ +│ (Go/Fiber) │ │ (master + 2 replicas │ +└──────┬───────┘ │ + sentinel) │ + │ └──────────▲───────────┘ + ▼ │ profile/rule reads +┌──────────────┐ │ (dns replica) +│ MongoDB │ ┌──────────┴───┐ ┌──────────────────┐ +│ │◀──│ DNS Proxy │──▶│ Recursors │ +└──────────────┘ │ (TLS term.) │ │ (sdns + Knot) │ + query logs & └──────▲───────┘ └──────────────────┘ + statistics │ DoH / DoT / DoQ + ┌──────┴───────┐ + │ DNS Clients │ + └──────────────┘ ``` +Additional services not shown above: + +- **blocklists** — periodically downloads and ingests filter lists (AdGuard, Hagezi, OISD, StevenBlack, …) into Redis/MongoDB. +- **dnscheck** — standalone DNS diagnostics microservice with GeoIP (MaxMind) lookup. + +Note that the DNS proxy terminates TLS for the encrypted DNS transports itself; Nginx only serves the web frontend. The API is exposed directly on port 3000. + ## Core Technologies **Backend Services** - Go & Fiber for high-performance APIs -- MongoDB for persistent storage of accounts, profiles, and telemetry -- Redis for caching and background jobs -- SDNS/Unbound for DNS resolution and policy enforcement +- MongoDB for persistent storage of accounts, profiles, query logs, and statistics +- Redis (master + two replicas + sentinel) for caching and distributing profile configuration to the proxy +- sdns and Knot Resolver as bundled recursors (Knot is the default) **Frontend** - React + TypeScript SPA (shadcn/ui & Radix UI component system) - Tailwind CSS for utility-first styling +- PWA with offline support and in-app update flow **Infrastructure** - Docker & Docker Compose for local orchestration -- Nginx as the public ingress & TLS termination layer +- Nginx serving the web frontend (the DNS proxy terminates TLS for encrypted DNS transports itself) - GitHub Actions for CI/CD automation ## Features **Encrypted DNS** -- DNS over HTTPS (DoH) -- DNS over TLS (DoT) -- DNS over QUIC (DoQ) +- DNS over HTTPS (DoH), DNS over TLS (DoT), DNS over QUIC (DoQ) +- Per-profile DNS stamps (`sdns://`) calculator and dnscrypt-proxy (via DoH) setup support +- DNSSEC validation (per profile, enabled by default) **Content Filtering** - Built-in blocklists (ads, malware, trackers) -- Custom allow/deny rules per profile +- Custom allow/deny rules per profile — domains and IPs, with rule groups and precedence +- Service-based blocking presets backed by ASN lookup (Google, Meta, TikTok, Netflix, …) +- DNS rebinding protection **User & Profile Management** - Multi-profile accounts with individualized policies -- MFA, email verification, and secure password workflows +- MFA (TOTP and WebAuthn/passkeys), email verification, and secure password workflows +- Profile settings export & import +- In-app announcements **Observability** -- Near real-time DNS query logging -- Exportable analytics for auditing +- Near real-time DNS query logging with outcome classification and quick-rule creation from log entries (opt-in, configurable retention from 1 hour to 1 month) +- Statistics and exportable analytics for auditing +- Prometheus metrics exposed by the proxy **Apple Device Integration** - Managed `.mobileconfig` profiles @@ -95,12 +101,27 @@ modDNS is built as a microservices architecture with the following components: ### Prerequisites - Docker & Docker Compose - Make (for the provided automation scripts) -- Node.js 18+ and npm (for the React application) -- Go 1.25+ (for backend services) +- Node.js 22+ and npm (for the React application) +- Go 1.25.8+ (toolchain version pinned in the `go.mod` files) +- Python 3.11 (for the backend E2E tests) - mkcert (optional, for trusted local TLS certificates) ### Quick Start +Before the first `make up`, create the required (gitignored) environment files from the tracked samples and provide the GeoLite2 databases: + +```bash +# 1. Environment files +cp api/.env.sample api/.env +cp proxy/.env.sample proxy/.env +cp dnscheck/.env.sample dnscheck/.env + +# 2. MaxMind GeoLite2 databases (mounted by the proxy and dnscheck) +# Place them under bootstrap/GeoLite2-ASN/ and bootstrap/GeoLite2-City/ +``` + +Then: + ```bash make up # Build and start every service stack make down # Stop and remove containers @@ -120,6 +141,9 @@ Certificates for local HTTPS access live in `certs/`. You can either generate th | `proxy/` | DNS proxy implementation | `tests/` | Integration and regression suites (pytest + testcontainers) | `bootstrap/`, `compose.*.yml` | Docker-compose orchestration and bootstrap assets +| `certs/` | Development TLS certificates and local CA +| `scripts/` | Helper scripts +| `.github/` | CI workflows (GitHub Actions), lint configs, issue/PR templates ## Development Workflow @@ -132,6 +156,7 @@ npm run lint npm run tsc npm run build ``` +`npm run dev` sources `app/env/.env.local`, which is gitignored — create it first (see the tracked `app/env/.env.production`, `.env.staging`, and `.env.test` for reference). ### API service (`api/`) ```bash @@ -139,29 +164,31 @@ cd api go mod tidy make test ``` -(See `api/Makefile` for additional targets like `make lint`, `make dev`, etc.) +(See `api/Makefile` for additional targets like `make lint`, `make gow` (live reload), `make swag`, and `make mockery`. From the repo root, `make dev_api` runs live reload inside the running container.) ### Proxy service (`proxy/`) ```bash -cd api +cd proxy go mod tidy make test ``` -(See `proxy/Makefile` for additional targets like `make lint`, `make dev`, etc.) +(See `proxy/Makefile` for additional targets like `make lint`, `make gow`, and `make mockery`. From the repo root, `make dev_proxy` runs live reload inside the running container.) ### Backend E2E tests (`tests/`) ```bash -python -m venv tests/venv -source tests/venv/bin/activate -pip install -r tests/requirements.txt -make test_ci # spins up containers via testcontainers +cd tests +python3.11 -m venv venv +source venv/bin/activate +make install_test_dependencies +make test_ci # spins up the stack via testcontainers +make test_failover # destructive Redis failover tests (excluded from test_ci, run separately) ``` ## Testing - **Web client**: `npm run lint && npm run tsc && npm run test` (unit) and `npm run test:e2e` (Playwright) - **Go services**: `go test ./...` inside each Go module (`api/`, `blocklists/`, `proxy/`, etc.) -- **Integration**: `source tests/venv/bin/activate && make test_ci` +- **Integration**: `cd tests && source venv/bin/activate && make test_ci` - **Static analysis**: `make lint` in relevant directories (Go linters + ESLint) ## Contributing