diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8197dc9..70aab88 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -106,9 +106,9 @@ jobs: path: TestResults/ if-no-files-found: warn - # PaperlessUI.Blazor is currently a scaffold-only project (no - # Paperless code). Once a real Paperless page is implemented, - # re-add it to Paperless.slnx and reintroduce the build step here. + # PaperlessUI.Blazor is in Paperless.slnx, so it's compiled as part + # of the NUKE build above — no separate step needed. Its Dockerfile + + # paperless-blazor compose service serve it behind nginx at /. frontend-angular: name: Build (PaperlessUI.Angular) diff --git a/AGENTS.md b/AGENTS.md index ec7c443..22ad4b0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # Paperless — repo guide for Claude -Document management with OCR, AI summarization, and full-text search. .NET 10 backend. Frontend story: React is the canonical/priority implementation, Angular is a parallel implementation kept for comparison, Blazor is currently a WIP scaffold. The production demo UI today is the vanilla SPA at `PaperlessREST/wwwroot/` which nginx serves from `compose.yaml`. +Document management with OCR, AI summarization, and full-text search. .NET 10 backend. Frontend story: the production demo UI served by `compose.yaml`/nginx (at `/`) is **PaperlessUI.Blazor** — an Interactive-Server Blazor port of the original `PaperlessREST/wwwroot/` SPA, which it replaced at the nginx root (drag-drop upload, live OCR + AI-summary over SSE, SignalR circuit). React is the canonical/priority frontend implementation and Angular a parallel one kept for stack comparison; both consume the same `/api/*` backend and are built/validated in CI but are not the deployed compose UI. The `wwwroot/` SPA still ships (served by the REST app's `UseStaticFiles`) as the original Blazor ported from. This file is the on-disk source of truth for working in this repo. Read it before touching anything. @@ -13,20 +13,21 @@ This file is the on-disk source of truth for working in this repo. Read it befor ``` Paperless.slnx # modern slnx; flat — NUKE 10 doesn't traverse wrappers ├── PaperlessREST/ # ASP.NET Core API (REST + SSE) -│ ├── wwwroot/ # Vanilla Bootstrap SPA — production demo UI mounted by nginx +│ ├── wwwroot/ # Original vanilla Bootstrap SPA (still served via UseStaticFiles); Blazor is the port that took over nginx / │ └── sample-data/ # XML batch fixtures (input/archive/error), mounted by compose ├── PaperlessServices/ # BackgroundService worker (OCR + GenAI) ├── PaperlessREST.Tests/ # xUnit v3 + Testcontainers ├── PaperlessServices.Tests/ # xUnit v3 + Testcontainers +├── Paperless.TestSupport/ # Shared test lib — ContainerFixtureBase (template-method) + builders + TestPdf + AsyncCleanup +├── PaperlessUI.Blazor/ # Blazor Web App (Interactive Server) — production demo UI behind nginx /; in slnx, compiled by backend CI, has Dockerfile + paperless-blazor compose service ├── PaperlessUI.React/ # Vite + React 19 + TS (canonical frontend) — PaperlessUI.React.esproj ├── PaperlessUI.Angular/ # Angular 21 + pnpm (parallel implementation) — PaperlessUI.Angular.esproj -├── PaperlessUI.Blazor/ # Blazor Web App scaffold — WIP, NOT in Paperless.slnx, NOT built by CI ├── Pipeline/ # NUKE build (Build.csproj) ├── docker/, compose.yaml └── docs/99_Reference/Rating-Matrix/ # course grading rubric (PDF + xlsx) ``` -React + Angular share the backend so the same use-cases can be compared across stacks. Blazor is on-disk but not building until a Paperless page is implemented (add back to Paperless.slnx + ci.yml when it does). +React + Angular share the backend so the same use-cases can be compared across stacks. Blazor is the deployed demo UI (a vanilla port of the wwwroot SPA): it's in `Paperless.slnx`, compiled by the backend CI job, and its Dockerfile + `paperless-blazor` compose service sit behind nginx at `/`. ## Build & test @@ -43,12 +44,12 @@ NUKE-based, single entry point. Targets compose via `Pipeline/Components/*.cs` ( The `ReportCoverage` gate is report-only (thresholds 0/0). CI publishes the markdown summary and Codecov diff; regressions surface in PR review, not as a hard build fail. -UI projects build via their respective toolchains, never via NUKE: +The React/Angular UIs are `esproj` and build via their pnpm toolchains, never via NUKE. Blazor is a `.csproj` in the slnx, so `./build.sh Compile` builds it like any backend project: ```bash cd PaperlessUI.React && pnpm install --frozen-lockfile && pnpm dev # canonical cd PaperlessUI.Angular && pnpm install --frozen-lockfile && pnpm start # parallel impl -# Blazor scaffold: dotnet run --project PaperlessUI.Blazor (WIP, not in slnx) +dotnet run --project PaperlessUI.Blazor # deployed demo UI (also compiled by ./build.sh Compile) ``` ## CI @@ -61,6 +62,8 @@ cd PaperlessUI.Angular && pnpm install --frozen-lockfile && pnpm start # par | `Build (PaperlessUI.Angular)` | non-blocking | `pnpm install --frozen-lockfile && pnpm run build` (ng's default config is production — do NOT pass `--configuration production` after `--`; pnpm 10 passes `--` literally to scripts) | | `Build (PaperlessUI.React)` | non-blocking | `pnpm install --frozen-lockfile && pnpm run build` | +`PaperlessUI.Blazor` is a `.csproj` in the slnx, so the backend job compiles it as part of the NUKE build — there is no separate Blazor CI job; its Dockerfile + `paperless-blazor` compose service serve it behind nginx at `/`. + The workflow declares `concurrency:` (cancel-in-progress on PRs only) and least-privilege `permissions:` (read defaults; codecov/check annotations escalate as needed). Coverage uploads to https://codecov.io/gh/ANcpLua/Paperless via tokenless OIDC. `codecov.yml` ignores host entry points, EF migrations, the pipeline, and test projects so the score reflects production surface only. @@ -107,7 +110,7 @@ Anti-patterns that fail the self-check: | Category | Where | |---|---| | Use Cases / REST API | `PaperlessREST/Features/DocumentManagement/Presentation/Endpoints/` | -| Web Frontend | React (`PaperlessUI.React/`, canonical) + Angular (`PaperlessUI.Angular/`, parallel impl) + the vanilla SPA at `PaperlessREST/wwwroot/` that nginx serves in production. Blazor (`PaperlessUI.Blazor/`) is a WIP scaffold, not currently built. | +| Web Frontend | **Blazor** (`PaperlessUI.Blazor/`) — the production demo UI nginx serves at `/` (Interactive-Server port of the wwwroot SPA). React (`PaperlessUI.React/`, canonical) + Angular (`PaperlessUI.Angular/`, parallel impl) share the backend for stack comparison. The original `PaperlessREST/wwwroot/` SPA still ships via `UseStaticFiles`. | | Queues | `SWEN3.Paperless.RabbitMq` consumed by REST + Services | | Logging | `Microsoft.Extensions.Logging`; `FakeLogger` in tests | | Validation | Mapster + DataAnnotations + FluentValidation at the boundary | diff --git a/README.md b/README.md index 424228e..3b2f373 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ # Paperless **Document management with OCR, AI summarization, and full-text search.** -.NET 10 backend · React + Angular frontends (Blazor scaffold WIP) · NUKE + xUnit v3 + Testcontainers. +.NET 10 backend · Blazor demo UI (behind nginx) + React/Angular frontends · NUKE + xUnit v3 + Testcontainers. CI @@ -24,20 +24,21 @@ Paperless.slnx # MSBuild slnx (modern format) ├── PaperlessServices/ # Background worker (OCR + GenAI) ├── PaperlessREST.Tests/ # xUnit v3 (unit + integration via Testcontainers) ├── PaperlessServices.Tests/ # xUnit v3 (unit + integration) -├── PaperlessREST/wwwroot/ # Vanilla Bootstrap SPA — production demo UI served by nginx +├── Paperless.TestSupport/ # Shared test lib (ContainerFixtureBase + builders + TestPdf + AsyncCleanup) +├── PaperlessREST/wwwroot/ # Original vanilla Bootstrap SPA (served via UseStaticFiles); Blazor took over nginx / +├── PaperlessUI.Blazor/ # Blazor Web App (Interactive Server) — production demo UI behind nginx /; in slnx, built by CI +│ └── PaperlessUI.Blazor.csproj ├── PaperlessUI.React/ # Frontend variant — Vite 8 + React 19 + TypeScript (canonical) │ └── PaperlessUI.React.esproj ├── PaperlessUI.Angular/ # Frontend variant — Angular 21 + pnpm (parallel impl) │ └── PaperlessUI.Angular.esproj -├── PaperlessUI.Blazor/ # Frontend variant — Blazor Web App scaffold (WIP, not in slnx, not built by CI) -│ └── PaperlessUI.Blazor.csproj ├── Pipeline/ # NUKE build (`./build.sh `) ├── docker/ # nginx config (single file: docker/nginx.conf) ├── PaperlessREST/sample-data/ # XML batch fixtures (input/archive/error), mounted by compose └── compose.yaml # Local stack (postgres, minio, rabbitmq, elastic) ``` -The production demo today is the vanilla SPA at `PaperlessREST/wwwroot/` (mounted by nginx in `compose.yaml`). React and Angular are parallel SDK-stack implementations consuming the same `/api/*` surface; Blazor is on-disk but not currently building. +The production demo UI is **PaperlessUI.Blazor** (Interactive Server), which nginx serves at `/` in `compose.yaml` — a vanilla-Blazor port of the original `PaperlessREST/wwwroot/` SPA that it replaced at the root. React and Angular are parallel SDK-stack implementations consuming the same `/api/*` surface (built/validated in CI, not deployed in compose). The `wwwroot/` SPA still ships via `UseStaticFiles`. Contributor & agent conventions live in [`AGENTS.md`](AGENTS.md) (`CLAUDE.md` is a symlink to it). @@ -63,8 +64,8 @@ cd PaperlessUI.React && pnpm install --frozen-lockfile && pnpm dev # Angular — parallel implementation cd PaperlessUI.Angular && pnpm install --frozen-lockfile && pnpm start -# Blazor scaffold (WIP, not currently in Paperless.slnx) -# dotnet run --project PaperlessUI.Blazor +# Blazor — deployed demo UI (in slnx; also compiled by ./build.sh Compile) +dotnet run --project PaperlessUI.Blazor ``` ## Architecture @@ -73,12 +74,12 @@ cd PaperlessUI.Angular && pnpm install --frozen-lockfile && pnpm start %%{init: {'theme':'dark'}}%% flowchart LR subgraph Clients - wwwroot[PaperlessREST/wwwroot
vanilla SPA — production] + Blazor[PaperlessUI.Blazor
demo UI — nginx /] React[PaperlessUI.React
canonical] Angular[PaperlessUI.Angular
parallel impl] - Blazor[PaperlessUI.Blazor
WIP scaffold] + wwwroot[PaperlessREST/wwwroot
original SPA — UseStaticFiles] end - wwwroot & React & Angular & Blazor -->|HTTPS / SSE| REST[PaperlessREST
ASP.NET Core] + Blazor & React & Angular & wwwroot -->|HTTPS / SSE| REST[PaperlessREST
ASP.NET Core] REST -->|EF Core| PG[(PostgreSQL)] REST -->|S3| MIN[(MinIO)] REST -->|HTTP| ES[(Elasticsearch)] @@ -92,7 +93,7 @@ flowchart LR ## CI + Coverage -`Build & Test` (gate): backend unit + integration + coverage gate + Codecov upload. +`Build & Test` (gate): backend unit + integration + coverage gate + Codecov upload (also compiles `PaperlessUI.Blazor`, which is in the slnx). Two non-gating jobs build the Angular and React apps via `pnpm`. Coverage uploads to https://codecov.io/gh/ANcpLua/Paperless via tokenless OIDC. @@ -105,7 +106,7 @@ The course rubric in [`docs/99_Reference/Rating-Matrix/`](docs/99_Reference/Rati | Category | Where it lives | |---|---| | **Use Cases / REST API** | `PaperlessREST/Features/DocumentManagement/Presentation/Endpoints/DocumentEndpoints.cs` | -| **Web Frontend** | `PaperlessREST/wwwroot/` (vanilla SPA, production demo) + `PaperlessUI.React/` (canonical) + `PaperlessUI.Angular/` (parallel impl). `PaperlessUI.Blazor/` is a WIP scaffold, currently out of build. | +| **Web Frontend** | `PaperlessUI.Blazor/` (Interactive Server) — the production demo UI nginx serves at `/` + `PaperlessUI.React/` (canonical) + `PaperlessUI.Angular/` (parallel impl). The original `PaperlessREST/wwwroot/` SPA still ships via `UseStaticFiles`. | | **Queues** | `SWEN3.Paperless.RabbitMq` package consumed by REST + Services | | **Logging** | `Microsoft.Extensions.Logging` everywhere; `FakeLogger` in tests | | **Validation** | Mapster + DataAnnotations + FluentValidation at the boundary | @@ -125,7 +126,7 @@ The course rubric in [`docs/99_Reference/Rating-Matrix/`](docs/99_Reference/Rati | Backend | Frontends | Infra | |---|---|---| -| .NET 10, ASP.NET Core, EF Core 10.0.x, Mapster, ErrorOr, Hangfire 1.8.23, Polly | React 19.2 + Vite 8 + TypeScript 6 (canonical), Angular 21 + pnpm 10 (parallel), Blazor Web App scaffold (WIP) | PostgreSQL 17, RabbitMQ 4.3, MinIO (date-pinned), Elasticsearch 9.1, nginx | +| .NET 10, ASP.NET Core, EF Core 10.0.x, Mapster, ErrorOr, Hangfire 1.8.23, Polly | Blazor Web App / Interactive Server (deployed demo UI), React 19.2 + Vite 8 + TypeScript 6 (canonical), Angular 21 + pnpm 10 (parallel) | PostgreSQL 17, RabbitMQ 4.3, MinIO (date-pinned), Elasticsearch 9.1, nginx | | xUnit v3.2.x, MTP v2, Testcontainers, AwesomeAssertions, Moq | – | OrbStack / Docker Compose | (Exact pin values live in `Version.props` and `Directory.Packages.props` — the table is commentary.) diff --git a/docs/superpowers/plans/2026-06-04-blazor-document-pipeline.md b/docs/superpowers/plans/2026-06-04-blazor-document-pipeline.md deleted file mode 100644 index 9c0bf1b..0000000 --- a/docs/superpowers/plans/2026-06-04-blazor-document-pipeline.md +++ /dev/null @@ -1,1110 +0,0 @@ -# PaperlessUI.Blazor Live Document Pipeline — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. -> -> **Project rule override:** This repo's CLAUDE.md says **"Do not write tests."** This plan therefore has **no test tasks**. Each task is verified by the repo's Validation hierarchy instead: clean `./build.sh Compile`, running the real compose stack, and Playwright for the live-streaming behavior. - -**Goal:** Replace the stock Blazor scaffold with a clean, vanilla-Blazor port of the `wwwroot/` demo SPA — drag-drop PDF upload, live OCR + AI-summary via **server-side `System.Net.ServerSentEvents.SseParser`**, list, search, delete — served same-origin behind nginx. - -**Architecture:** Blazor Web App, Interactive Server. The Blazor *server* consumes the REST API's two SSE streams (`/api/v1/ocr-results`, `/api/v1/events/genai`) over the internal network with `SseParser`, then pushes UI updates to the browser over the SignalR circuit (`InvokeAsync(StateHasChanged)`). No browser `EventSource`, no JS interop for events, no CORS. nginx fronts both the Blazor app (`/`) and the API (`/api/`). - -**Tech Stack:** .NET 10.0.300, `ANcpLua.NET.Sdk.Web` 3.4.41, Blazor Interactive Server, `System.Net.ServerSentEvents` 10.0.8 (out-of-band package), Bootstrap 5.3.2 + bootstrap-icons (CDN), nginx, Docker Compose. - -**Branch:** `feat/blazor-document-pipeline` (spec committed at `6f56e6d`; `Paperless.slnx` Blazor line already staged in the working tree). - ---- - -## File Structure - -| File | Responsibility | -|---|---| -| `Directory.Packages.props` (modify) | CPM version pin for `System.Net.ServerSentEvents`. | -| `PaperlessUI.Blazor.csproj` (modify) | `PackageReference` to `System.Net.ServerSentEvents`. | -| `Models/Documents.cs` (create) | Transport records mirroring the REST contract (decoupled copy). | -| `Services/PaperlessApiClient.cs` (create) | Typed `HttpClient` — list / upload / search / delete. | -| `Services/DocumentEventStream.cs` (create) | Scoped per-circuit SSE consumer; raises `OnChanged`, exposes `Connected`. | -| `Program.cs` (modify) | DI: named + typed HttpClients, scoped event stream, `Paperless:ApiBaseUrl`. | -| `appsettings.json` (modify) | Default `Paperless:ApiBaseUrl`. | -| `Components/App.razor` (modify) | Bootstrap + icons via CDN (no local `lib/`). | -| `Components/DocumentCard.razor` (create) | One document card (status, OCR preview, summary, delete). | -| `Components/Pages/Home.razor` (rewrite) | The page: theme wrapper, upload, search, toasts, list, banner. | -| `Components/Layout/NavMenu.razor` (modify) | Trim Counter/Weather links. | -| `Components/Pages/Counter.razor`, `Weather.razor` (delete) | Template cruft. | -| `wwwroot/app.css` (modify) | A few app-specific rules. | -| `PaperlessUI.Blazor/Dockerfile` (create) | Multi-stage build; custom-SDK COPY order preserved. | -| `compose.yaml` (modify) | Add `paperless-blazor`; remove `wwwroot` mount; nginx `depends_on`. | -| `docker/nginx.conf` (modify) | `paperless-blazor` upstream; `/` + `/_blazor` WebSocket; drop static root. | - ---- - -### Task 1: Pin `System.Net.ServerSentEvents` (CPM) - -**Files:** -- Modify: `Directory.Packages.props` -- Modify: `PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` - -- [ ] **Step 1: Add the central version pin** - -In `Directory.Packages.props`, inside the existing `` of `` entries, add (keep the list alphabetical if it is): - -```xml - -``` - -(`10.0.8` matches the repo's .NET 10.0.8 runtime family; it's the newest version already in the local NuGet cache.) - -- [ ] **Step 2: Reference it from the Blazor project** - -Replace the body of `PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` with: - -```xml - - - - true - false - - - - - - - -``` - -- [ ] **Step 3: Restore and confirm the package resolves** - -Run: `dotnet restore PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: restore succeeds, no NU1102/NU1101 for `System.Net.ServerSentEvents`. - -- [ ] **Step 4: Commit** - -```bash -git add Directory.Packages.props PaperlessUI.Blazor/PaperlessUI.Blazor.csproj -git commit -m "build: add System.Net.ServerSentEvents to PaperlessUI.Blazor" -``` - ---- - -### Task 2: Transport models - -**Files:** -- Create: `PaperlessUI.Blazor/Models/Documents.cs` - -- [ ] **Step 1: Create the models** - -```csharp -namespace PaperlessUI.Blazor.Models; - -/// Document metadata as returned by GET /api/v1/documents (items) and /{id}. -public sealed record DocumentDto -{ - public required Guid Id { get; init; } - public required string FileName { get; init; } - public required string Status { get; init; } - public required DateTimeOffset CreatedAt { get; init; } - public DateTimeOffset? ProcessedAt { get; init; } - public string? Content { get; init; } - public string? Summary { get; init; } - public DateTimeOffset? SummaryGeneratedAt { get; init; } -} - -/// One hit from GET /api/v1/documents/search. -public sealed record DocumentSearchResultDto -{ - public required Guid Id { get; init; } - public required string FileName { get; init; } - public string? Content { get; init; } - public string? Summary { get; init; } - public required DateTimeOffset CreatedAt { get; init; } - public required string Status { get; init; } -} - -/// Cursor-paginated wrapper from GET /api/v1/documents. -public sealed record PaginatedDocumentsResponse -{ - public required List Items { get; init; } - public Guid? NextCursor { get; init; } - public bool HasMore { get; init; } -} - -/// 202 body from POST /api/v1/documents. -public sealed record CreateDocumentResponse -{ - public required Guid Id { get; init; } - public required string FileName { get; init; } - public required string Status { get; init; } - public required DateTimeOffset CreatedAt { get; init; } -} -``` - -- [ ] **Step 2: Build** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds. - -- [ ] **Step 3: Commit** - -```bash -git add PaperlessUI.Blazor/Models/Documents.cs -git commit -m "feat(blazor): document transport models" -``` - ---- - -### Task 3: PaperlessApiClient (typed HttpClient) - -**Files:** -- Create: `PaperlessUI.Blazor/Services/PaperlessApiClient.cs` - -- [ ] **Step 1: Create the client** - -```csharp -using System.Net.Http.Headers; -using System.Net.Http.Json; -using System.Text.Json; -using Microsoft.AspNetCore.Components.Forms; -using PaperlessUI.Blazor.Models; - -namespace PaperlessUI.Blazor.Services; - -/// REST calls to PaperlessREST's document API. One instance per request (transient typed client). -public sealed class PaperlessApiClient(HttpClient http) -{ - public const long MaxUploadBytes = 50L * 1024 * 1024; // matches nginx client_max_body_size 50m - - private static readonly JsonSerializerOptions Json = new(JsonSerializerDefaults.Web); - - public async Task> GetDocumentsAsync(CancellationToken ct = default) - { - var page = await http.GetFromJsonAsync("/api/v1/documents", Json, ct); - return page?.Items ?? []; - } - - public async Task> SearchAsync(string query, int limit = 50, CancellationToken ct = default) - { - var url = $"/api/v1/documents/search?query={Uri.EscapeDataString(query)}&limit={limit}"; - var hits = await http.GetFromJsonAsync>(url, Json, ct); - return hits?.ConvertAll(static h => new DocumentDto - { - Id = h.Id, - FileName = h.FileName, - Status = h.Status, - CreatedAt = h.CreatedAt, - Content = h.Content, - Summary = h.Summary - }) ?? []; - } - - public async Task<(CreateDocumentResponse? Doc, string? Error)> UploadAsync(IBrowserFile file, CancellationToken ct = default) - { - using var content = new MultipartFormDataContent(); - await using var stream = file.OpenReadStream(MaxUploadBytes, ct); - using var fileContent = new StreamContent(stream); - fileContent.Headers.ContentType = new MediaTypeHeaderValue("application/pdf"); - content.Add(fileContent, "file", file.Name); - - using var resp = await http.PostAsync("/api/v1/documents", content, ct); - if (resp.IsSuccessStatusCode) - { - var created = await resp.Content.ReadFromJsonAsync(Json, ct); - return (created, null); - } - - var body = await resp.Content.ReadAsStringAsync(ct); - return (null, $"{(int)resp.StatusCode} {resp.ReasonPhrase}: {body}"); - } - - public async Task DeleteAsync(Guid id, CancellationToken ct = default) - { - using var resp = await http.DeleteAsync($"/api/v1/documents/{id}", ct); - return resp.IsSuccessStatusCode; - } -} -``` - -- [ ] **Step 2: Build** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds. - -- [ ] **Step 3: Commit** - -```bash -git add PaperlessUI.Blazor/Services/PaperlessApiClient.cs -git commit -m "feat(blazor): typed REST client for documents" -``` - ---- - -### Task 4: DocumentEventStream (server-side SSE consumer) - -**Files:** -- Create: `PaperlessUI.Blazor/Services/DocumentEventStream.cs` - -- [ ] **Step 1: Create the SSE consumer** - -```csharp -using System.Net.ServerSentEvents; - -namespace PaperlessUI.Blazor.Services; - -/// -/// Consumes PaperlessREST's two SSE streams server-side with the .NET BCL — -/// the mirror of how SWEN3.Paperless.RabbitMq produces them via TypedResults.ServerSentEvents. -/// Scoped per Interactive Server circuit; raises on every event so the page -/// can refetch + re-render. Reconnects with backoff on drop. -/// -public sealed class DocumentEventStream(IHttpClientFactory factory, ILogger logger) - : IAsyncDisposable -{ - public event Action? OnChanged; - - /// True while at least the OCR stream is connected; drives the "disconnected" banner. - public bool Connected { get; private set; } - - private CancellationTokenSource? _cts; - - public void Start() - { - if (_cts is not null) return; - _cts = new CancellationTokenSource(); - // Backoff intervals mirror the old wwwroot/app.js (5s OCR, 10s GenAI). - _ = ConsumeAsync("/api/v1/ocr-results", TimeSpan.FromSeconds(5), isPrimary: true, _cts.Token); - _ = ConsumeAsync("/api/v1/events/genai", TimeSpan.FromSeconds(10), isPrimary: false, _cts.Token); - } - - private async Task ConsumeAsync(string path, TimeSpan retry, bool isPrimary, CancellationToken ct) - { - var client = factory.CreateClient("paperless"); - while (!ct.IsCancellationRequested) - { - try - { - using var resp = await client.GetAsync(path, HttpCompletionOption.ResponseHeadersRead, ct); - resp.EnsureSuccessStatusCode(); - if (isPrimary) { Connected = true; OnChanged?.Invoke(); } - - await using var stream = await resp.Content.ReadAsStreamAsync(ct); - var parser = SseParser.Create(stream); - await foreach (SseItem item in parser.EnumerateAsync(ct)) - { - logger.LogInformation("SSE {Event} on {Path}", item.EventType, path); - OnChanged?.Invoke(); - } - } - catch (OperationCanceledException) when (ct.IsCancellationRequested) - { - break; - } - catch (Exception ex) - { - if (isPrimary) { Connected = false; OnChanged?.Invoke(); } - logger.LogWarning(ex, "SSE {Path} dropped; reconnecting in {Seconds}s", path, retry.TotalSeconds); - try { await Task.Delay(retry, ct); } - catch (OperationCanceledException) { break; } - } - } - } - - public async ValueTask DisposeAsync() - { - if (_cts is null) return; - await _cts.CancelAsync(); - _cts.Dispose(); - _cts = null; - } -} -``` - -- [ ] **Step 2: Build** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds (confirms `SseParser`/`SseItem` resolve from the package). - -- [ ] **Step 3: Commit** - -```bash -git add PaperlessUI.Blazor/Services/DocumentEventStream.cs -git commit -m "feat(blazor): server-side SSE consumer via SseParser" -``` - ---- - -### Task 5: DI wiring + config - -**Files:** -- Modify: `PaperlessUI.Blazor/Program.cs` -- Modify: `PaperlessUI.Blazor/appsettings.json` - -- [ ] **Step 1: Set the default API base URL** - -Replace `PaperlessUI.Blazor/appsettings.json` with: - -```json -{ - "Logging": { - "LogLevel": { - "Default": "Information", - "Microsoft.AspNetCore": "Warning" - } - }, - "AllowedHosts": "*", - "Paperless": { - "ApiBaseUrl": "http://localhost:8080" - } -} -``` - -(In compose this is overridden by the env var `Paperless__ApiBaseUrl=http://paperless-rest:8080`.) - -- [ ] **Step 2: Register clients + event stream** - -Replace `PaperlessUI.Blazor/Program.cs` with: - -```csharp -using PaperlessUI.Blazor.Components; -using PaperlessUI.Blazor.Services; - -var builder = WebApplication.CreateBuilder(args); - -// Add services to the container. -builder.Services.AddRazorComponents() - .AddInteractiveServerComponents(); - -var apiBaseUrl = builder.Configuration["Paperless:ApiBaseUrl"] ?? "http://localhost:8080"; -var apiBase = new Uri(apiBaseUrl); - -// Named client used by the long-lived SSE consumer; typed client for short REST calls. -builder.Services.AddHttpClient("paperless", c => c.BaseAddress = apiBase); -builder.Services.AddHttpClient(c => c.BaseAddress = apiBase); -builder.Services.AddScoped(); - -var app = builder.Build(); - -// Configure the HTTP request pipeline. -if (!app.Environment.IsDevelopment()) -{ - app.UseExceptionHandler("/Error", createScopeForErrors: true); - app.UseHsts(); -} -app.UseStatusCodePagesWithReExecute("/not-found", createScopeForStatusCodePages: true); -app.UseHttpsRedirection(); - -app.UseAntiforgery(); - -app.MapStaticAssets(); -app.MapRazorComponents() - .AddInteractiveServerRenderMode(); - -app.Run(); -``` - -- [ ] **Step 3: Build** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds. - -- [ ] **Step 4: Commit** - -```bash -git add PaperlessUI.Blazor/Program.cs PaperlessUI.Blazor/appsettings.json -git commit -m "feat(blazor): DI + config for API client and SSE stream" -``` - ---- - -### Task 6: Bootstrap via CDN (App.razor) - -**Files:** -- Modify: `PaperlessUI.Blazor/Components/App.razor` - -(There is no local `wwwroot/lib/bootstrap`, so the scaffold's `@Assets["lib/bootstrap/..."]` link 404s. Use the same CDN the old SPA used.) - -- [ ] **Step 1: Swap the stylesheet links** - -In `Components/App.razor`, replace this line: - -```razor - -``` - -with: - -```razor - - -``` - -Leave the `app.css` and `PaperlessUI.Blazor.styles.css` links and everything else unchanged. - -- [ ] **Step 2: Commit** - -```bash -git add PaperlessUI.Blazor/Components/App.razor -git commit -m "feat(blazor): load Bootstrap + icons via CDN" -``` - ---- - -### Task 7: DocumentCard component - -**Files:** -- Create: `PaperlessUI.Blazor/Components/DocumentCard.razor` - -- [ ] **Step 1: Create the component** - -```razor -@using PaperlessUI.Blazor.Models - -
-
-
- -
@Doc.FileName
-
- -
-
-

- Uploaded: @Format(Doc.CreatedAt) - @if (Doc.ProcessedAt is not null) - { -
- - @($"Processed: {Format(Doc.ProcessedAt)}") - } -

- - - @if (Doc.Status == "Pending") - { - - } - @Doc.Status - - - @if (Doc.Status == "Completed" && !string.IsNullOrEmpty(Doc.Content)) - { -
-
OCR Text Preview
-
- @Preview(Doc.Content) -
-
- } - - @if (!string.IsNullOrEmpty(Doc.Summary)) - { -
-
AI Summary
-
- @Doc.Summary -
-
- } -
-
- -@code { - [Parameter, EditorRequired] public required DocumentDto Doc { get; set; } - [Parameter] public EventCallback OnDelete { get; set; } - - private string StatusColor => Doc.Status switch - { - "Pending" => "warning", - "Completed" => "success", - _ => "danger" - }; - - private Task OnDeleteClicked() => OnDelete.InvokeAsync(Doc.Id); - - private static string Format(DateTimeOffset? value) => - value?.LocalDateTime.ToString("g") ?? "Unknown"; - - private static string Preview(string content) => - content.Length > 500 ? content[..500] + "…" : content; -} -``` - -- [ ] **Step 2: Build** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds. - -- [ ] **Step 3: Commit** - -```bash -git add PaperlessUI.Blazor/Components/DocumentCard.razor -git commit -m "feat(blazor): document card component" -``` - ---- - -### Task 8: Home page (upload + search + live list + toasts) - -**Files:** -- Rewrite: `PaperlessUI.Blazor/Components/Pages/Home.razor` - -- [ ] **Step 1: Rewrite the page** - -```razor -@page "/" -@rendermode InteractiveServer -@using Microsoft.AspNetCore.Components.Forms -@using PaperlessUI.Blazor.Models -@using PaperlessUI.Blazor.Services -@inject PaperlessApiClient Api -@inject DocumentEventStream Events -@implements IAsyncDisposable - -Paperless OCR System - -
- - - @if (!Events.Connected) - { -
- Live Updates Disconnected -
- } - -

Paperless OCR System

- -
- -

Upload PDF Documents

-

Drag and drop PDF files here or click to browse

- -
- -
- - - - -
- -
- -
-

Documents

- -
- - @if (_documents.Count == 0) - { -
- -

@(string.IsNullOrEmpty(_activeSearch) ? "No documents uploaded yet" : "No documents match your search")

-
- } - else - { - @foreach (var doc in _documents) - { - - } - } -
- -
- @foreach (var toast in _toasts) - { -
- @toast.Message - -
- } -
- -@code { - private readonly List _documents = []; - private readonly List _toasts = []; - private string _searchText = ""; - private string _activeSearch = ""; - private string _theme = "light"; - - private sealed record Toast(string Message, string Level); - - protected override async Task OnInitializedAsync() - { - Events.OnChanged += HandleChanged; - Events.Start(); - await ReloadAsync(); - } - - private void ToggleTheme() => _theme = _theme == "dark" ? "light" : "dark"; - - // SSE callback — runs off the render thread, so marshal onto the circuit. - // Signature matches DocumentEventStream's `event EventHandler? OnChanged` (CA1003-clean). - private void HandleChanged(object? sender, EventArgs e) => _ = InvokeAsync(async () => - { - await ReloadAsync(); - StateHasChanged(); - }); - - private async Task ReloadAsync() - { - try - { - var docs = string.IsNullOrEmpty(_activeSearch) - ? await Api.GetDocumentsAsync() - : await Api.SearchAsync(_activeSearch); - _documents.Clear(); - _documents.AddRange(docs); - } - catch (Exception ex) - { - Notify($"Failed to load documents: {ex.Message}", "danger"); - } - } - - private async Task Refresh() - { - await ReloadAsync(); - StateHasChanged(); - } - - private async Task OnFilesSelected(InputFileChangeEventArgs e) - { - foreach (var file in e.GetMultipleFiles(maximumFileCount: 20)) - { - if (!string.Equals(file.ContentType, "application/pdf", StringComparison.OrdinalIgnoreCase)) - { - Notify($"{file.Name} is not a PDF", "warning"); - continue; - } - - var (doc, error) = await Api.UploadAsync(file); - if (error is not null) - { - Notify($"Failed: {error}", "danger"); - continue; - } - - if (doc is not null && !_documents.Exists(d => d.Id == doc.Id)) - { - _documents.Insert(0, new DocumentDto - { - Id = doc.Id, FileName = doc.FileName, Status = doc.Status, CreatedAt = doc.CreatedAt - }); - } - - Notify($"Uploaded {file.Name}", "success"); - } - - StateHasChanged(); - } - - private async Task RunSearch() - { - _activeSearch = _searchText.Trim(); - await ReloadAsync(); - StateHasChanged(); - } - - private async Task ClearSearch() - { - _searchText = ""; - _activeSearch = ""; - await ReloadAsync(); - StateHasChanged(); - } - - private async Task OnSearchKey(KeyboardEventArgs e) - { - if (e.Key == "Enter") await RunSearch(); - } - - private async Task Delete(Guid id) - { - if (await Api.DeleteAsync(id)) - { - _documents.RemoveAll(d => d.Id == id); - Notify("Document deleted", "success"); - } - else - { - Notify("Delete failed", "danger"); - } - StateHasChanged(); - } - - private void Notify(string message, string level) - { - var toast = new Toast(message, level); - _toasts.Add(toast); - _ = Task.Delay(TimeSpan.FromSeconds(4)).ContinueWith(_ => - InvokeAsync(() => { _toasts.Remove(toast); StateHasChanged(); })); - } - - public ValueTask DisposeAsync() - { - Events.OnChanged -= HandleChanged; - return ValueTask.CompletedTask; // the scoped DocumentEventStream is disposed with the circuit - } -} -``` - -- [ ] **Step 2: Build** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds. - -- [ ] **Step 3: Commit** - -```bash -git add PaperlessUI.Blazor/Components/Pages/Home.razor -git commit -m "feat(blazor): document pipeline page with live SSE updates" -``` - ---- - -### Task 9: Scaffold cleanup - -**Files:** -- Delete: `PaperlessUI.Blazor/Components/Pages/Counter.razor` -- Delete: `PaperlessUI.Blazor/Components/Pages/Weather.razor` -- Modify: `PaperlessUI.Blazor/Components/Layout/NavMenu.razor` -- Modify: `PaperlessUI.Blazor/wwwroot/app.css` - -- [ ] **Step 1: Delete template pages** - -```bash -git rm PaperlessUI.Blazor/Components/Pages/Counter.razor PaperlessUI.Blazor/Components/Pages/Weather.razor -``` - -- [ ] **Step 2: Trim NavMenu** - -Replace the `` block in `Components/Layout/NavMenu.razor` so only Home remains: - -```razor - -``` - -- [ ] **Step 3: Add app-specific CSS** - -Append to `PaperlessUI.Blazor/wwwroot/app.css`: - -```css -.min-w-0 { min-width: 0; } -``` - -- [ ] **Step 4: Build to confirm no dangling references** - -Run: `dotnet build PaperlessUI.Blazor/PaperlessUI.Blazor.csproj` -Expected: build succeeds (no references to the deleted `Counter`/`Weather` routes remain). - -- [ ] **Step 5: Commit** - -```bash -git add PaperlessUI.Blazor/Components/Layout/NavMenu.razor PaperlessUI.Blazor/wwwroot/app.css -git commit -m "chore(blazor): drop template Counter/Weather pages, trim nav" -``` - ---- - -### Task 10: Dockerfile - -**Files:** -- Create: `PaperlessUI.Blazor/Dockerfile` - -- [ ] **Step 1: Create the Dockerfile** - -(Modeled on `PaperlessREST/Dockerfile`. The custom-SDK COPY order is mandatory — see the repo gotcha. No `Paperless.Contracts` reference: the Blazor app carries its own DTOs.) - -```dockerfile -FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base -USER root -RUN apt-get update && apt-get install -y tzdata && \ - ln -sf /usr/share/zoneinfo/Europe/Vienna /etc/localtime && \ - echo "Europe/Vienna" > /etc/timezone && \ - apt-get clean && rm -rf /var/lib/apt/lists/* -ENV TZ=Europe/Vienna - -USER app -WORKDIR /app -EXPOSE 8080 - -FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build -ARG BUILD_CONFIGURATION=Release -ENV DOCKER_BUILD=true -WORKDIR /src - -# Root MSBuild + NuGet config must be present BEFORE restore so the custom -# ANcpLua.NET.Sdk.Web resolver and CPM versions resolve. -COPY ["global.json", "nuget.config", "Directory.Packages.props", "Version.props", "./"] - -COPY ["PaperlessUI.Blazor/PaperlessUI.Blazor.csproj", "PaperlessUI.Blazor/"] -RUN dotnet restore "PaperlessUI.Blazor/PaperlessUI.Blazor.csproj" - -COPY ["PaperlessUI.Blazor/", "PaperlessUI.Blazor/"] -WORKDIR "/src/PaperlessUI.Blazor" -RUN dotnet build "PaperlessUI.Blazor.csproj" -c $BUILD_CONFIGURATION -o /app/build -p:DOCKER_BUILD=true - -FROM build AS publish -ARG BUILD_CONFIGURATION=Release -RUN dotnet publish "PaperlessUI.Blazor.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false -p:DOCKER_BUILD=true - -FROM base AS final -WORKDIR /app -COPY --from=publish /app/publish . -ENV ASPNETCORE_ENVIRONMENT=Production -ENTRYPOINT ["dotnet", "PaperlessUI.Blazor.dll"] -``` - -- [ ] **Step 2: Build the image to verify the COPY order** - -Run: `docker build -f PaperlessUI.Blazor/Dockerfile -t paperless-blazor:dev .` -Expected: image builds; no "Could not resolve SDK ANcpLua.NET.Sdk.Web" error. - -- [ ] **Step 3: Commit** - -```bash -git add PaperlessUI.Blazor/Dockerfile -git commit -m "build(blazor): Dockerfile preserving custom-SDK restore order" -``` - ---- - -### Task 11: compose.yaml — add the Blazor service - -**Files:** -- Modify: `compose.yaml` - -- [ ] **Step 1: Add the `paperless-blazor` service** - -Under `# APPLICATION SERVICES`, after the `paperless-rest` service block, add: - -```yaml - paperless-blazor: - container_name: paperless-blazor - image: ${PAPERLESS_BLAZOR_IMAGE:-paperless-blazor:latest} - build: - context: . - dockerfile: PaperlessUI.Blazor/Dockerfile - depends_on: - paperless-rest: { condition: service_started } - aspire-dashboard: { condition: service_started } - environment: - ASPNETCORE_ENVIRONMENT: ${ASPNETCORE_ENVIRONMENT:-Development} - ASPNETCORE_URLS: http://+:8080 - Paperless__ApiBaseUrl: http://paperless-rest:8080 - # OTel env mirrors the sibling services for consistency (no OTel SDK is - # wired in any app yet, so these are currently inert — kept aligned). - OTEL_EXPORTER_OTLP_ENDPOINT: http://aspire-dashboard:18889 - OTEL_SERVICE_NAME: paperless-blazor - TZ: ${TZ} -``` - -- [ ] **Step 2: Point nginx at the Blazor app and drop the static mount** - -In the `nginx:` service, change `volumes:` to mount only the config (remove the `wwwroot` html mount), and add `paperless-blazor` to `depends_on`: - -```yaml - nginx: - container_name: paperless-nginx - image: ${NGINX_IMAGE} - ports: - - "80:80" - volumes: - - ./docker/nginx.conf:/etc/nginx/nginx.conf:ro - depends_on: - paperless-rest: { condition: service_started } - paperless-blazor: { condition: service_started } - environment: - TZ: ${TZ} - healthcheck: - test: [ "CMD", "nginx", "-t" ] - interval: 5s - timeout: 3s - retries: 10 -``` - -- [ ] **Step 3: Validate compose syntax** - -Run: `docker compose config >/dev/null && echo OK` -Expected: `OK` (no YAML/interpolation errors). - -- [ ] **Step 4: Commit** - -```bash -git add compose.yaml -git commit -m "build(compose): add paperless-blazor service, serve it via nginx" -``` - ---- - -### Task 12: nginx.conf — route `/` to Blazor with WebSocket - -**Files:** -- Modify: `docker/nginx.conf` - -- [ ] **Step 1: Add the Blazor upstream** - -After the existing `upstream paperless-api { … }` block, add: - -```nginx - upstream paperless-blazor { - server paperless-blazor:8080; - } -``` - -- [ ] **Step 2: Replace the static-root config with a proxy to Blazor** - -Remove these static-file lines from the `server { … }` block: - -```nginx - # Static files (index.html, app.js, etc.) - root /usr/share/nginx/html; - index index.html; - - # Main app - location = / { - try_files /index.html =404; - } - - # Static assets - location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ { - expires 1h; - add_header Cache-Control "public, immutable"; - } -``` - -and replace them with a single proxy location that handles the Blazor SignalR WebSocket (`/_blazor`) and all app assets: - -```nginx - # Blazor app (Interactive Server) — root + SignalR circuit + static assets - location / { - proxy_pass http://paperless-blazor; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_buffering off; - proxy_read_timeout 100s; - } -``` - -Leave the `/api/`, SSE, `/hangfire`, `/docs`, `/openapi`, and `/health` location blocks unchanged. (The two SSE blocks are now only hit server-to-server, bypassing nginx, but are harmless to keep.) - -- [ ] **Step 3: Validate nginx config syntax** - -Run: `docker run --rm -v "$PWD/docker/nginx.conf:/etc/nginx/nginx.conf:ro" nginx:alpine nginx -t` -Expected: `syntax is ok` / `test is successful`. - -- [ ] **Step 4: Commit** - -```bash -git add docker/nginx.conf -git commit -m "build(nginx): proxy / to paperless-blazor with WebSocket upgrade" -``` - ---- - -### Task 13: Solution build + commit slnx - -**Files:** -- Already staged: `Paperless.slnx` (Blazor project line) - -- [ ] **Step 1: Full solution build via NUKE** - -Run: `./build.sh Compile` -Expected: build succeeds with `PaperlessUI.Blazor` included (it's now in `Paperless.slnx`). Zero errors. - -- [ ] **Step 2: Commit the slnx wiring** - -```bash -git add Paperless.slnx -git commit -m "build: add PaperlessUI.Blazor to Paperless.slnx" -``` - ---- - -### Task 14: End-to-end validation (run the real stack + Playwright) - -**Files:** none (validation only) - -- [ ] **Step 1: Bring up the full stack** - -Run: `docker compose up -d --build` -Then wait for health: `docker compose ps` until `paperless-rest`, `paperless-blazor`, `nginx` are up. - -- [ ] **Step 2: Smoke-test the UI loads through nginx** - -Run: `curl -fsS http://localhost/ | grep -i "Paperless OCR System" && echo PAGE_OK` -Expected: `PAGE_OK` (Blazor page served at `/` via nginx). - -- [ ] **Step 3: Playwright — upload + live streaming** - -Drive a browser through the live flow (use the Playwright MCP tools): -1. Navigate to `http://localhost/`. -2. Set the file input to a sample PDF: `PaperlessREST/sample-data/input/*.pdf`. -3. Assert a card appears with status `Pending`. -4. Wait (poll up to ~60s) for the card's badge to become `Completed` **without reloading the page** — proves the OCR SSE → circuit push works. -5. Wait for the "AI Summary" section to render — proves the GenAI SSE path. -6. Screenshot the streamed result. - -Expected: status transitions Pending → Completed live; summary appears live. - -- [ ] **Step 4: Tear down** - -Run: `docker compose down` - -- [ ] **Step 5: Push the branch and open a PR** - -```bash -git push -u origin feat/blazor-document-pipeline -gh pr create --base main --head feat/blazor-document-pipeline \ - --title "PaperlessUI.Blazor: live document pipeline (vanilla Blazor port of wwwroot SPA)" \ - --body "Replaces the Blazor scaffold with a clean port of the wwwroot demo SPA: drag-drop PDF upload, live OCR + AI-summary via server-side System.Net.ServerSentEvents.SseParser, list, search, delete. Served same-origin behind nginx. See docs/superpowers/specs/2026-06-04-blazor-document-pipeline-design.md." -``` - -- [ ] **Step 6: Watch CI** - -Run: `gh pr checks --watch` -Expected: `Build & Test (backend)` green (now compiles Blazor via the slnx). - ---- - -## Self-Review - -**Spec coverage:** -- Upload / live OCR+summary / list / search / delete / disconnected banner / theme toggle / refresh / Hangfire link → Tasks 7–8. ✓ -- Server-side `SseParser` consumer → Task 4. ✓ -- Replace wwwroot SPA at `/` → Tasks 11–12. ✓ -- Keep aspire-dashboard; Blazor exports matching OTel env → Task 11 (with the inert-OTel finding noted). ✓ -- Dockerfile w/ custom-SDK COPY order → Task 10. ✓ -- slnx + CI → Task 13 (slnx already staged; CI covered because Blazor is now in the solution NUKE builds). ✓ -- Validation = build + run + Playwright (no tests) → Tasks 13–14. ✓ - -**Placeholder scan:** No TBD/TODO; every code step shows full content. ✓ - -**Type consistency:** `DocumentDto`/`CreateDocumentResponse`/`PaginatedDocumentsResponse`/`DocumentSearchResultDto` defined in Task 2 and used consistently in Tasks 3, 7, 8. `DocumentEventStream.OnChanged`/`Connected`/`Start()` defined in Task 4 and used in Task 8. `PaperlessApiClient` method names (`GetDocumentsAsync`/`SearchAsync`/`UploadAsync`/`DeleteAsync`) consistent across Tasks 3 and 8. ✓ - -**Open items deferred (by design, in spec's Out-of-scope):** list pagination UI, theme persistence across reloads, fixing `wwwroot/app.js`'s paginated-list bug, targeted per-document SSE updates. diff --git a/docs/superpowers/specs/2026-06-04-blazor-document-pipeline-design.md b/docs/superpowers/specs/2026-06-04-blazor-document-pipeline-design.md deleted file mode 100644 index 5d4222c..0000000 --- a/docs/superpowers/specs/2026-06-04-blazor-document-pipeline-design.md +++ /dev/null @@ -1,196 +0,0 @@ -# PaperlessUI.Blazor — live document pipeline (design) - -**Date:** 2026-06-04 -**Status:** Draft for review -**Author:** Claude + ancplua - -## Goal - -Turn the `PaperlessUI.Blazor` scaffold (currently the stock Home/Counter/Weather -template) into a clean, vanilla-Blazor port of the production demo UI — the two files -`PaperlessREST/wwwroot/index.html` + `wwwroot/app.js`. Same feature set, but driven from -C# instead of hand-rolled JS, and using modern .NET as the "interesting" part. - -The signature move: consume the backend's live SSE streams **server-side** with -`System.Net.ServerSentEvents.SseParser` — the exact mirror of how the -`SWEN3.Paperless.RabbitMq` library *produces* them (`TypedResults.ServerSentEvents`). -Producer and consumer both use the .NET 10 BCL SSE API, opposite directions. - -Modeled structurally on the Microsoft Agent Framework `AgentWebChat.Web` sample: -single rich page, Interactive Server, streaming-driven, no third-party component library. - -## Scope (locked decisions) - -- **SSE consumption:** server-side `SseParser` in the Blazor circuit. No browser - `EventSource`, no JS interop for events. -- **UI surface:** the Blazor app **replaces** the vanilla `wwwroot` SPA as what nginx - serves at `/`. The old `index.html`/`app.js` stay in the repo as reference but are no - longer mounted by nginx. -- **Observability:** `aspire-dashboard` stays. It is wired and live (both app services - export OTel to it). The new Blazor service will also export OTel to it. -- **Render mode:** Interactive Server (unchanged from the scaffold). -- **No component library:** vanilla HTML + the existing Bootstrap CSS, matching today's SPA. - -## Feature set (parity with `app.js`) - -1. Drag-and-drop / file-picker **PDF upload** (multipart `file`). -2. **Live status:** a card flips `Pending → Completed/Failed` when OCR finishes, then the - **AI summary** appears when GenAI finishes — both pushed live via SSE. -3. **Document list** of cards (filename, upload/processed time, status badge + spinner, - OCR text preview, AI summary, delete). -4. **Full-text search** + clear. -5. **Delete** a document. -6. **"Live updates disconnected"** banner when the SSE loop drops (with auto-reconnect). -7. Light/dark theme toggle (Bootstrap `data-bs-theme`), refresh button, Hangfire link. - -## Backend contract (authoritative — from `DocumentEndpoints.cs`) - -| Verb | Route | Returns | -|---|---|---| -| GET | `/api/v1/documents` | `PaginatedDocumentsResponse { items[], hasMore, nextCursor }` | -| GET | `/api/v1/documents/search?query=&limit=` | `DocumentSearchResultDto[]` | -| GET | `/api/v1/documents/{id}` | `DocumentDto` | -| GET | `/api/v1/documents/{id}/summary` | `SummaryDto { summary }` | -| POST | `/api/v1/documents` (multipart `file`) | `202` `CreateDocumentResponse` | -| DELETE | `/api/v1/documents/{id}` | `204` | - -SSE (produced by the library via `MapSse`): - -| Stream | Events | Data | -|---|---|---| -| `/api/v1/ocr-results` | `ocr-completed`, `ocr-failed` | `OcrEvent` (jobId, documentId, …) | -| `/api/v1/events/genai` | `genai-completed`, `genai-failed` | `GenAIEvent` (documentId, summary, errorMessage) | - -> **Finding / rot to confirm:** `wwwroot/app.js` does `const docs = await response.json(); -> docs.forEach(...)` against `GET /api/v1/documents`, but that endpoint now returns the -> **paginated object** `{ items, hasMore, nextCursor }` (added in #42), not a bare array. -> The current SPA's list fetch is therefore likely broken against `main`. The Blazor port -> will use the **current** contract and read `.items`. Worth fixing `app.js` separately, -> but out of scope here. - -## Architecture - -``` -Browser ──HTTP/WS──▶ nginx :80 ─┬─ "/" ▶ paperless-blazor:8080 (Interactive Server circuit) - ├─ "/_blazor" ▶ paperless-blazor:8080 (SignalR WebSocket) - ├─ "/api/" ▶ paperless-rest:8080 - └─ /hangfire,/docs,/openapi,/health ▶ paperless-rest:8080 - -paperless-blazor ──server-to-server (internal net, NOT via nginx)──▶ paperless-rest:8080 - • REST: list / upload / search / delete - • SSE : SseParser over /api/v1/ocr-results + /api/v1/events/genai -``` - -Because SSE is consumed server-side (Blazor server → REST, internal network), there is -**no browser SSE connection and no CORS** to configure. The only browser↔server channel -is the Blazor SignalR circuit. nginx's existing SSE `location` blocks become unused by -the UI (kept for now; optional cleanup later). - -## Components (each small, single-purpose) - -- **`Components/Pages/Home.razor`** — the page. Upload zone, search bar, document grid. - Subscribes to `DocumentEventStream`; on any event → refetch list → `InvokeAsync(StateHasChanged)`. - Implements `IAsyncDisposable` to tear down subscriptions. -- **`Components/DocumentCard.razor`** — one document. Razor version of `documentCardTemplate`: - status badge + spinner, OCR preview (first 500 chars), AI summary section, delete button. -- **`Services/PaperlessApiClient.cs`** — typed `HttpClient`. Methods: `GetDocumentsAsync`, - `UploadAsync(IBrowserFile)`, `SearchAsync(query, limit)`, `DeleteAsync(id)`. Uses - `JsonSerializerDefaults.Web` (camelCase). Base address from config. -- **`Services/DocumentEventStream.cs`** — the SSE consumer. Two long-running loops (OCR + - GenAI), each: `HttpClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead, ct)` → - `SseParser.Create(stream)` → `await foreach (SseItem item in parser.EnumerateAsync(ct))`. - Raises a C# `event` (e.g. `OnDocumentChanged`). Reconnect with backoff (5s OCR / 10s GenAI, - matching `app.js`) on stream end/error. Exposes a `Connected` flag for the disconnected banner. -- **`Models/DocumentDto.cs`** (+ search/summary/event DTOs) — mirror the REST contract. - -> Threading note (Interactive Server): SSE callbacks run off the render thread. All UI -> mutation must go through `InvokeAsync(StateHasChanged)`. The event-stream loops are tied -> to the circuit and cancelled on component dispose. - -## Data flow (upload → live) - -1. User drops a PDF → `PaperlessApiClient.UploadAsync` → `POST /api/v1/documents` → `202`. -2. Card appears immediately as `Pending` (optimistic, from the upload response). -3. OCR worker finishes → REST publishes `ocr-completed` → Blazor server's `SseParser` yields - it → `OnDocumentChanged` → page refetches list → card flips to `Completed`, OCR preview shows. -4. GenAI finishes → `genai-completed` → refetch → AI summary section fills in. -5. Failures (`ocr-failed` / `genai-failed`) → refetch (status reflects `Failed`) + toast. - -(Refetch-on-event matches `app.js`. Targeted single-document updates by `documentId` from the -event payload are a possible later optimization, not in v1.) - -## Error handling - -- **Upload:** client checks `.pdf`; server returns `400` (validation), `503` - (ServiceUnavailable + Retry-After), or `500`. Each surfaces a Bootstrap toast. -- **SSE drop:** loop catches, sets `Connected=false` (banner shows), waits backoff, reconnects. -- **List/search/delete failure:** toast; list left intact. -- **Circuit drop:** the scaffold's `ReconnectModal` already handles SignalR reconnection. - -## Infra changes - -### New: `PaperlessUI.Blazor/Dockerfile` -Multi-stage, modeled on `PaperlessREST/Dockerfile`. **Must** COPY `global.json`, -`nuget.config`, `Directory.Packages.props`, `Version.props` into the build context **before** -`dotnet restore` — the custom `ANcpLua.NET.Sdk.Web` SDK resolver fails otherwise (documented -repo gotcha). Final image runs `ASPNETCORE_URLS=http://+:8080`. - -### `compose.yaml` -Add service `paperless-blazor`: -- `build`: context `.`, dockerfile `PaperlessUI.Blazor/Dockerfile`. -- `environment`: `Paperless__ApiBaseUrl=http://paperless-rest:8080`, - `ASPNETCORE_URLS=http://+:8080`, `OTEL_EXPORTER_OTLP_ENDPOINT=http://aspire-dashboard:18889`, - `OTEL_SERVICE_NAME=paperless-blazor`, `TZ`. -- `depends_on`: `paperless-rest` (started), `aspire-dashboard` (started). -- `nginx.depends_on` gains `paperless-blazor`. -- Remove the `./PaperlessREST/wwwroot:/usr/share/nginx/html:ro` mount (Blazor now serves the UI). - -### `docker/nginx.conf` -- Add `upstream paperless-blazor { server paperless-blazor:8080; }`. -- `location / { proxy_pass http://paperless-blazor; }` with `Upgrade`/`Connection "upgrade"`, - `proxy_http_version 1.1` (needed for the Blazor SignalR WebSocket at `/_blazor`). -- Drop the static-root config (`root`, `index`, `try_files /index.html`, the static-asset - `location ~* \.(js|css|...)`) — Blazor serves its own assets through `/`. -- Keep `/api/`, `/hangfire`, `/docs`, `/openapi`, `/health` → `paperless-api` as-is. -- SSE `location` blocks: leave for now (unused by UI; harmless). - -### `PaperlessUI.Blazor/Program.cs` -- Register `PaperlessApiClient` + `DocumentEventStream` as typed `HttpClient`s - (`builder.Services.AddHttpClient<…>(c => c.BaseAddress = config["Paperless:ApiBaseUrl"])`). -- Add OpenTelemetry OTLP export (match the two existing services) so traces land in the dashboard. -- Keep `AddRazorComponents().AddInteractiveServerComponents()` + `MapRazorComponents()`. - -### Scaffold cleanup -- Delete template pages `Counter.razor`, `Weather.razor`; repurpose `Home.razor`. -- Trim `NavMenu.razor` (remove Counter/Weather links). -- Keep `Error.razor`, `NotFound.razor`, `ReconnectModal.razor`, `App.razor`. - -### `Paperless.slnx` + CI -- Add `PaperlessUI.Blazor.csproj` to `Paperless.slnx` (flat — no `` wrapper, per the - NUKE 10 slnx gotcha). NUKE `Compile` then builds it as part of the solution. -- `.github/workflows/ci.yml`: the backend job's `Compile` now covers Blazor (it's in the slnx). - No separate UI job needed (unlike the pnpm React/Angular jobs). This satisfies the repo - guide's "add Blazor back to slnx + ci.yml when a page is implemented." - -## Validation (per global rules, in priority order) - -1. **Playwright e2e:** `docker compose up` → open `http://localhost` → upload a sample PDF - from `PaperlessREST/sample-data/input` → assert the card flips Pending→Completed and the - AI summary appears (live, no manual refresh). Screenshot the streamed result. -2. **Run the real artifact:** the compose stack end to end (above). -3. **Clean build floor:** `./build.sh Compile` zero errors/warnings with Blazor in the slnx. - -No tests written (global rule: do not write tests). - -## Out of scope (v1) - -- List pagination UI ("load more" / cursor) — fetch first page only; note the `hasMore` flag. -- Auth, document detail route, batch-job UI (Hangfire link remains). -- Fixing `wwwroot/app.js`'s paginated-list bug (flagged above; separate task). -- Targeted per-document SSE updates (refetch-on-event is v1). - -## Git / workspace note - -Work proceeds on a dedicated feature branch (`feat/blazor-document-pipeline`). The working -tree had ~20 unrelated pre-existing modified files + 1 untracked test file at session start; -those are **not** touched by this work — only files created/changed for this feature are staged. diff --git a/docs/superpowers/specs/2026-06-04-test-infrastructure-uplift-design.md b/docs/superpowers/specs/2026-06-04-test-infrastructure-uplift-design.md deleted file mode 100644 index b8ca6cd..0000000 --- a/docs/superpowers/specs/2026-06-04-test-infrastructure-uplift-design.md +++ /dev/null @@ -1,1031 +0,0 @@ -# Test-Infrastructure Uplift — Design Spec - -Date: 2026-06-04 -Scope: `PaperlessREST.Tests`, `PaperlessServices.Tests`, new `Paperless.TestSupport` -Kind: **refactor only** — no new test CASES, no behavior change. Build must stay 0/0 under `./build.sh Compile`; both test projects must still compile. - ---- - -## 0. Goal & guardrails - -De-duplicate the two integration-test fixtures (`SharedRestContainerFixture`, `SharedContainerFixture`) and their satellite helpers behind a small shared class library `Paperless.TestSupport`, using a template-method base fixture. Remove genuinely dead infrastructure (the REST Elasticsearch container — see §7). Replace REST env-var mutation with in-memory config injection (mirroring how the Services fixture already does it). - -**Hard constraints (verified against source, must hold after the change):** - -1. Every member in the SURFACE `mustPreserve` list keeps its exact name, signature, accessibility, and (for `IClassFixture`/`ICollectionFixture`) its public parameterless constructor. - - `SharedRestContainerFixture` stays `public sealed`, `IAsyncLifetime`, with `Client`/`Services`/`DbFactory` public getters and `CreateAsyncScope()`. - - `SharedContainerFixture` stays `public`, `IAsyncLifetime`, with `Services`, `UploadPdfAsync(string)`, `WaitForDocumentAsync(…)`, `WaitForSearchResultsAsync(…)`. - - `SharedContainerCollection` keeps `[CollectionDefinition(Name)] : ICollectionFixture` and `const string Name = "SharedContainer"`. -2. `DatabaseFixture` (REST) is OUT OF SCOPE for the base-class refactor. It may only adopt the new `TestEnv`/`TestContainers` helpers for its `.env.test` load and Postgres builder (§6); its public surface (`Services`, `ContextFactory`, `LogCollector`, `CreateAsyncScope`) is untouched. -3. `[assembly: CaptureConsole]` / `[assembly: CaptureTrace]` stay **exactly one pair per test assembly** and **must NOT move into `Paperless.TestSupport`** (assembly-level attributes attach to the assembly they are compiled into; moving them would attach them to the library and silently stop capturing in the test assemblies). They stay in the respective fixture files. -4. `Paperless.TestSupport` references **only** infrastructure packages. It must NOT reference `PaperlessREST.csproj` or `PaperlessServices.csproj` (WAF `` and `Host.CreateApplicationBuilder` composition stay in the derived fixtures). -5. No new `[Fact]`/`[Theory]`. The five consumer test files change only where a fixture member's *call site* is unaffected — ideally they do not change at all, except `DocumentEndpointTests` whose manual cleanup list is replaced by the RAII helper (§5, behavior-equivalent). - -**Evidence that recalibrated the input analysis (do not skip — it changes the plan):** - -- The SHARED/PATTERNS analysis claims `WaitForLogAsync`/`WaitForLogCountAsync` are "cross-consumed" by `PaperlessREST.Tests/Unit/ListenerLifecycleTests.cs`. **This is false.** `ListenerLifecycleTests.cs:477` defines its own `private static async Task WaitForLogAsync(FakeLogCollector source, Func predicate, CancellationToken ct)` — a *different signature* from the Services extension `WaitForLogAsync(this FakeLogCollector, Func,bool>, TimeSpan?, TimeSpan?, CancellationToken)`. The only consumer of the *extension* wait helpers is `PaperlessServices.Tests/Unit/OcrWorkerTests.cs`. Therefore moving the wait helpers to the shared lib is *proportionate convenience* (both assemblies could use them), not load-bearing — and we will move them, but we will NOT touch `ListenerLifecycleTests`' private method (out of scope, would be a behavioral risk for zero benefit). -- `GetFullLoggerText` IS genuinely shared: 8 consumer files in `PaperlessREST.Tests`, 7 in `PaperlessServices.Tests`, byte-identical bodies (only an XML doc comment differs). Safe to centralize. -- REST tests reference `ElasticsearchClient` **nowhere** except the `Document` type alias in `PaperlessREST.Tests/GlobalUsings.cs:69` (which exists *to disambiguate away from* `Elastic.Clients.Elasticsearch.Document`). No REST test queries Elastic. The REST ES container is dead weight → **remove it** (§7), do not generalize ES into the base. -- `Paperless.slnx` already lists every project flatly and the NUKE `Compile` target builds `GetSolutionPath()` (the whole solution). Adding `Paperless.TestSupport` to `Paperless.slnx` is sufficient for CI to build it — no `Pipeline/` change needed. - ---- - -## 1. File-by-file plan (overview) - -| Path | Action | Responsibility | -|---|---|---| -| `Paperless.TestSupport/Paperless.TestSupport.csproj` | NEW | net10.0 lib, `ANcpLua.NET.Sdk`, infra-only package refs | -| `Paperless.TestSupport/GlobalUsings.cs` | NEW | shared usings for the lib | -| `Paperless.TestSupport/TestEnv.cs` | NEW | `.env.test` load (once) + image-name-from-env-with-default | -| `Paperless.TestSupport/TestContainers.cs` | NEW | builder factories: Postgres/RabbitMq/Minio/Elasticsearch (configured, unstarted) | -| `Paperless.TestSupport/MinioBucket.cs` | NEW | create bucket + endpoint string helper | -| `Paperless.TestSupport/AsyncCleanup.cs` | NEW | RAII `IAsyncDisposable` (mirrors `SessionCleanup`) | -| `Paperless.TestSupport/FakeLoggerExtensions.cs` | NEW | single `GetFullLoggerText` + `WaitForLogAsync` + `WaitForLogCountAsync` | -| `Paperless.TestSupport/ContainerFixtureBase.cs` | NEW | abstract template-method base: shared container lifecycle + ES poll helpers + guarded dispose | -| `Paperless.slnx` | MODIFY | add `Paperless.TestSupport/Paperless.TestSupport.csproj` | -| `PaperlessREST.Tests/PaperlessREST.Tests.csproj` | MODIFY | `ProjectReference` TestSupport; drop now-shared package refs that are only in TestSupport? (NO — keep, see §8) | -| `PaperlessServices.Tests/PaperlessServices.Tests.csproj` | MODIFY | `ProjectReference` TestSupport | -| `PaperlessREST.Tests/FakeLoggerExtensions.cs` | DELETE | moved to TestSupport | -| `PaperlessServices.Tests/FakeLoggerExtensions.cs` | DELETE | moved to TestSupport | -| `PaperlessREST.Tests/GlobalUsings.cs` | MODIFY | add `global using Paperless.TestSupport;` | -| `PaperlessServices.Tests/GlobalUsings.cs` | MODIFY | add `global using Paperless.TestSupport;` | -| `PaperlessREST.Tests/Integration/SharedRestContainerFixture.cs` | MODIFY | derive from base; in-memory config; drop ES container; keep surface | -| `PaperlessServices.Tests/Integration/WorkerTestBase.cs` | MODIFY | derive from base; keep surface + collection def | -| `PaperlessREST.Tests/Integration/DatabaseFixture.cs` | MODIFY (light) | use `TestEnv` + `TestContainers.Postgres()`; surface untouched | -| `PaperlessREST.Tests/Integration/DocumentEndpointTests.cs` | MODIFY (light) | replace `_createdDocIds` manual cleanup with `AsyncCleanup` (behavior-equivalent) | - ---- - -## 2. `Paperless.TestSupport` project - -### 2.1 `Paperless.TestSupport/Paperless.TestSupport.csproj` (NEW) - -Plain `ANcpLua.NET.Sdk` (NOT `.Test` — it is a support library, not a test runner project; it must not pull MTP/xUnit-runner wiring). It is referenced by the two `.Test`/`.Web` test projects. `IsTestProject` is left unset. - -```xml - - - - net10.0 - Paperless.TestSupport - - false - - - - - - - - - - - - - - - - - - -``` - -> Version pins: every `PackageReference` above already has a `PackageVersion` in `Directory.Packages.props` (`Testcontainers.*`, `Minio`, `Elastic.Clients.Elasticsearch`, `DotNetEnv`, `Microsoft.Extensions.Diagnostics.Testing`, and **`xunit.v3.mtp-v2`** = `$(XunitV3MtpV2Version)`). **Resolved (review fix, was §9.1):** the lib references **`xunit.v3.mtp-v2`** (the repo's pinned, runner-free lineage) — NOT bare `xunit.v3` (no CPM entry → NU1604, and it would pull a second xunit core lineage). `Microsoft.Extensions.DependencyInjection.Abstractions` is **not** referenced explicitly — it flows transitively, so no new CPM entry is needed and restore stays clean. - -### 2.2 `Paperless.TestSupport/GlobalUsings.cs` (NEW) - -```csharp -global using System.Diagnostics.CodeAnalysis; - -global using DotNet.Testcontainers.Builders; -global using DotNet.Testcontainers.Containers; - -global using DotNetEnv; - -global using Elastic.Clients.Elasticsearch; - -global using Microsoft.Extensions.DependencyInjection; -global using Microsoft.Extensions.Logging.Testing; - -global using Minio; -global using Minio.DataModel.Args; - -global using Testcontainers.Elasticsearch; -global using Testcontainers.Minio; -global using Testcontainers.PostgreSql; -global using Testcontainers.RabbitMq; - -global using Xunit; -``` - -### 2.3 `Paperless.TestSupport/TestEnv.cs` (NEW) - -Centralizes the three duplicated `Env.TraversePath().Load(".env.test")` static-ctor blocks and the `Environment.GetEnvironmentVariable(name) ?? default` image-resolution idiom. - -```csharp -namespace Paperless.TestSupport; - -/// -/// One-time .env.test loading and environment-variable image resolution -/// shared by every integration fixture. Replaces the three duplicated static -/// constructors (REST shared fixture, REST DatabaseFixture, Services fixture). -/// -public static class TestEnv -{ - private static readonly object Gate = new(); - private static bool _loaded; - - /// - /// Loads .env.test exactly once per process via - /// . Idempotent and thread-safe so it can be - /// called from every fixture's static constructor without re-loading. - /// - public static void Load() - { - if (_loaded) return; - lock (Gate) - { - if (_loaded) return; - Env.TraversePath().Load(".env.test"); - _loaded = true; - } - } - - /// - /// Returns the container image for , falling back to - /// when the variable is unset. Mirrors the - /// Environment.GetEnvironmentVariable(...) ?? "image:tag" pattern the - /// fixtures duplicated per container. - /// - public static string Image(string envVar, string defaultImage) => - Environment.GetEnvironmentVariable(envVar) ?? defaultImage; -} -``` - -### 2.4 `Paperless.TestSupport/TestContainers.cs` (NEW) - -Single home for the four container builders. Each returns a **configured-but-unstarted** container. The Elasticsearch builder folds in the `xpack.security.http.ssl.enabled=false` fix from the Services fixture (the comment is preserved verbatim — it is load-bearing tribal knowledge per CLAUDE.md Gotchas). Default image tags are taken from the current source. - -```csharp -namespace Paperless.TestSupport; - -/// -/// Factory methods producing configured-but-unstarted Testcontainers. -/// Centralizes image defaults (overridable via env vars) and the -/// Elasticsearch TLS wait-strategy fix that both fixtures previously duplicated. -/// -public static class TestContainers -{ - private const string DefaultPostgresImage = "postgres:17-alpine"; - private const string DefaultRabbitmqImage = "rabbitmq:4.3.0-management"; - private const string DefaultMinioImage = "minio/minio:RELEASE.2025-09-07T16-13-09Z"; - private const string DefaultElasticsearchImage = - "docker.elastic.co/elasticsearch/elasticsearch:9.1.3"; - - public static PostgreSqlContainer Postgres() => - new PostgreSqlBuilder(TestEnv.Image("POSTGRES_IMAGE", DefaultPostgresImage)) - .WithWaitStrategy(Wait.ForUnixContainer() - .UntilMessageIsLogged("database system is ready to accept connections")) - .Build(); - - public static RabbitMqContainer RabbitMq() => - new RabbitMqBuilder(TestEnv.Image("RABBITMQ_IMAGE", DefaultRabbitmqImage)) - .Build(); - - public static MinioContainer Minio() => - new MinioBuilder(TestEnv.Image("MINIO_IMAGE", DefaultMinioImage)) - .Build(); - - public static ElasticsearchContainer Elasticsearch() => - new ElasticsearchBuilder(TestEnv.Image("ELASTIC_IMAGE", DefaultElasticsearchImage)) - .WithEnvironment("discovery.type", "single-node") - .WithEnvironment("xpack.security.enabled", "false") - // Required so Testcontainers' ElasticsearchConfiguration.TlsEnabled evaluates to false - // (it AND-s xpack.security.enabled with xpack.security.http.ssl.enabled). Without this, - // the built-in wait strategy probes HTTPS while ES listens on plain HTTP, and hangs. - .WithEnvironment("xpack.security.http.ssl.enabled", "false") - .WithEnvironment("ES_JAVA_OPTS", "-Xms512m -Xmx512m") - .WithEnvironment("bootstrap.memory_lock", "false") - .Build(); -} -``` - -> Note: the REST fixture previously set an explicit `.WithWaitStrategy(... "started")` on ES; that path is being deleted with the ES container (§7). The Services fixture relied on the default wait strategy + its own `WaitForElasticsearchAsync()` cluster-health poll. We keep the Services semantics: `TestContainers.Elasticsearch()` does NOT add an explicit message wait strategy (the `xpack...ssl.enabled=false` env makes the built-in strategy work), and the base fixture's ES readiness poll (§4.1) preserves `WaitForElasticsearchAsync`. - -### 2.5 `Paperless.TestSupport/MinioBucket.cs` (NEW) - -```csharp -namespace Paperless.TestSupport; - -/// -/// MinIO endpoint/bucket helpers shared by the fixtures. Wraps the -/// $"{Hostname}:{MappedPort}" endpoint string and the one-shot -/// MakeBucketAsync block both fixtures duplicated. -/// -public static class MinioBucket -{ - private const int MinioPort = 9000; - - /// Host:port endpoint string for a started MinIO container. - public static string Endpoint(MinioContainer minio) => - $"{minio.Hostname}:{minio.GetMappedPublicPort(MinioPort)}"; - - /// Creates against the started container. - public static async Task CreateBucketAsync(MinioContainer minio, string bucketName) - { - using MinioClient client = new(); - client - .WithEndpoint(Endpoint(minio)) - .WithCredentials(minio.GetAccessKey(), minio.GetSecretKey()) - .Build(); - await client.MakeBucketAsync(new MakeBucketArgs().WithBucket(bucketName)); - } -} -``` - -### 2.6 `Paperless.TestSupport/AsyncCleanup.cs` (NEW) - -Generic RAII teardown mirroring agent-framework's `SessionCleanup`. Used by `DocumentEndpointTests` to replace its manual `_createdDocIds` list. Kept minimal (YAGNI): a single `Func` invoked once on dispose. - -```csharp -namespace Paperless.TestSupport; - -/// -/// RAII per-test cleanup: await using var cleanup = new AsyncCleanup(() => ...); -/// runs the delegate on scope exit even when an assertion throws. Mirrors the -/// agent-framework SessionCleanup pattern. The delegate runs at most once. -/// -public sealed class AsyncCleanup(Func onDispose) : IAsyncDisposable -{ - private int _disposed; - - public async ValueTask DisposeAsync() - { - if (Interlocked.Exchange(ref _disposed, 1) != 0) return; - await onDispose(); - } -} -``` - -### 2.7 `Paperless.TestSupport/FakeLoggerExtensions.cs` (NEW) - -Single copy. `GetFullLoggerText` body is byte-identical to both current copies. `WaitForLogAsync`/`WaitForLogCountAsync` are lifted verbatim from the Services copy (the only one that has them). Namespace becomes `Paperless.TestSupport`; consumers pick it up via the added `global using` (§8.3). - -```csharp -namespace Paperless.TestSupport; - -public static class FakeLoggerExtensions -{ - /// - /// Gets full log text from the collector with optional formatting. - /// - public static string GetFullLoggerText( - this FakeLogCollector source, - Func? formatter = null) - { - StringBuilder sb = new(); - IReadOnlyList snapshot = source.GetSnapshot(); - formatter ??= record => $"{record.Level} - {record.Message}"; - - foreach (FakeLogRecord record in snapshot) - { - sb.AppendLine(formatter(record)); - } - - return sb.ToString(); - } - - /// - /// Waits for a log condition to be met, polling at regular intervals. - /// Returns true if condition was met, false if timeout expired. - /// - public static async Task WaitForLogAsync( - this FakeLogCollector source, - Func, bool> condition, - TimeSpan? timeout = null, - TimeSpan? pollInterval = null, - CancellationToken cancellationToken = default) - { - timeout ??= TimeSpan.FromSeconds(5); - pollInterval ??= TimeSpan.FromMilliseconds(25); - - using CancellationTokenSource cts = - CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); - cts.CancelAfter(timeout.Value); - - try - { - while (!cts.Token.IsCancellationRequested) - { - if (condition(source.GetSnapshot())) - { - return true; - } - - await Task.Delay(pollInterval.Value, cts.Token).ConfigureAwait(false); - } - } - catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested) - { - // Timeout expired, not user cancellation - } - - return condition(source.GetSnapshot()); // Final check - } - - /// - /// Waits for a specific number of log messages matching a predicate. - /// - public static Task WaitForLogCountAsync( - this FakeLogCollector source, - Func predicate, - int expectedCount, - TimeSpan? timeout = null, - CancellationToken cancellationToken = default) => - source.WaitForLogAsync( - logs => logs.Count(predicate) >= expectedCount, - timeout, - cancellationToken: cancellationToken); -} -``` - -> `StringBuilder` needs `System.Text.Both` current copies compile because each test project's `GlobalUsings.cs` (or the SDK's implicit usings) brings in `System.Text`. `Paperless.TestSupport` uses `ANcpLua.NET.Sdk` with implicit usings (the SDK enables `enable`), so `System.Text.StringBuilder`, `System.Threading`, `System.Threading.Tasks`, and `System.Linq` are in scope. If implicit usings are NOT enabled by the SDK, add `global using System.Text;` to `Paperless.TestSupport/GlobalUsings.cs`. (§9.2 open decision — verify by build.) - -### 2.8 `Paperless.TestSupport/ContainerFixtureBase.cs` (NEW) - -The template-method base. It owns the **shared** lifecycle only: declaring which containers to start, starting them in parallel, MinIO bucket creation, the ES readiness poll, the ES polling helpers, and guarded teardown. It does **not** know about `WebApplicationFactory`, `Host.CreateApplicationBuilder`, Postgres+`MapEnum`, config delivery, or `FakeTextSummarizer` — all of which stay in derived fixtures via the `ConfigureSutAsync` hook. - -Design choices: -- `protected abstract bool UsesPostgres { get; }` (REST=true, Services=false) controls whether a Postgres container is started; REST exposes `PostgresConnectionString` to its own `ConfigureSutAsync`. -- The ES polling helpers (`WaitForDocumentAsync`, `WaitForSearchResultsAsync`) live here so the Services fixture inherits them unchanged. They resolve `ElasticsearchClient` from `Services`, so they only work after `Services` is set — same as today. -- `Services` is `public` with `protected set` so derived fixtures assign it in `ConfigureSutAsync`. -- Guarded teardown adopts the Services-style try/swallow per the analysis recommendation; REST loses no correctness (its old unguarded path was strictly weaker). -- The base owns container fields and the per-fixture bucket/index name so both derived fixtures stop re-declaring them. - -```csharp -namespace Paperless.TestSupport; - -/// -/// Template-method base for the integration-test container fixtures. Owns the -/// shared container lifecycle (RabbitMQ + MinIO + Elasticsearch, plus an optional -/// Postgres), MinIO bucket creation, Elasticsearch readiness polling, the -/// Elasticsearch document/search polling helpers, and guarded teardown. -/// -/// Derived fixtures supply the system-under-test by overriding -/// (which must assign ) -/// and tear it down in . The base never references -/// PaperlessREST or PaperlessServices. -/// -/// -public abstract class ContainerFixtureBase : IAsyncLifetime -{ - private const int ElasticsearchPort = 9200; - - private readonly PostgreSqlContainer? _postgres; - private readonly RabbitMqContainer _rabbit = TestContainers.RabbitMq(); - private readonly MinioContainer _minio = TestContainers.Minio(); - private readonly ElasticsearchContainer _elastic = TestContainers.Elasticsearch(); - - protected ContainerFixtureBase() - { - _postgres = UsesPostgres ? TestContainers.Postgres() : null; - } - - /// Whether to start a Postgres container (REST = true, Services = false). - protected abstract bool UsesPostgres { get; } - - /// Unique per-fixture bucket name; the bucket is created during init. - protected string BucketName { get; } = $"test-{Guid.NewGuid():N}"; - - /// Unique per-fixture default Elasticsearch index name. - protected string IndexName { get; } = $"test_{Guid.NewGuid():N}"; - - /// MinIO host:port endpoint string (valid after containers start). - protected string MinioEndpoint => MinioBucket.Endpoint(_minio); - - protected string MinioAccessKey => _minio.GetAccessKey(); - protected string MinioSecretKey => _minio.GetSecretKey(); - protected string RabbitConnectionString => _rabbit.GetConnectionString(); - protected string ElasticsearchUri => - $"http://{_elastic.Hostname}:{_elastic.GetMappedPublicPort(ElasticsearchPort)}"; - - /// Postgres connection string; throws if is false. - protected string PostgresConnectionString => - (_postgres ?? throw new InvalidOperationException( - "This fixture did not request a Postgres container (UsesPostgres == false).")) - .GetConnectionString(); - - /// Service provider for the constructed SUT. Assigned by . - public IServiceProvider Services { get; protected set; } = null!; - - public async ValueTask InitializeAsync() - { - var starts = new List - { - _rabbit.StartAsync(), - _minio.StartAsync(), - _elastic.StartAsync() - }; - if (_postgres is not null) starts.Add(_postgres.StartAsync()); - await Task.WhenAll(starts); - - await WaitForElasticsearchAsync(); - await MinioBucket.CreateBucketAsync(_minio, BucketName); - - await ConfigureSutAsync(); - } - - public async ValueTask DisposeAsync() - { - // Guard SUT teardown and swallow per-container Dispose failures so that an - // exception thrown during InitializeAsync (e.g. a wait-strategy timeout) is - // not masked by a secondary NRE/dispose error during xUnit fixture cleanup. - try { await DisposeSutAsync(); } catch { /* best-effort */ } - - try { await _rabbit.DisposeAsync(); } catch { /* best-effort */ } - try { await _minio.DisposeAsync(); } catch { /* best-effort */ } - try { await _elastic.DisposeAsync(); } catch { /* best-effort */ } - if (_postgres is not null) - { - try { await _postgres.DisposeAsync(); } catch { /* best-effort */ } - } - } - - /// - /// Builds the system under test and assigns . - /// Runs after containers are started and the bucket is created. - /// - protected abstract ValueTask ConfigureSutAsync(); - - /// Tears down the system under test (host/factory). Default: no-op. - protected virtual ValueTask DisposeSutAsync() => ValueTask.CompletedTask; - - /// - /// Polls Elasticsearch until a document is found or timeout occurs. - /// Replaces brittle Task.Delay patterns with deterministic polling. - /// - public async Task> WaitForDocumentAsync( - string documentId, - CancellationToken cancellationToken, - TimeSpan? timeout = null, - TimeSpan? pollInterval = null) - { - timeout ??= TimeSpan.FromSeconds(10); - pollInterval ??= TimeSpan.FromMilliseconds(100); - - var client = Services.GetRequiredService(); - using CancellationTokenSource cts = new(timeout.Value); - using var linked = - CancellationTokenSource.CreateLinkedTokenSource(cts.Token, cancellationToken); - - while (!linked.Token.IsCancellationRequested) - { - var response = await client.GetAsync( - documentId, - g => g.Index(client.ElasticsearchClientSettings.DefaultIndex), - linked.Token); - - if (response.Found) - { - return response; - } - - await Task.Delay(pollInterval.Value, linked.Token); - } - - // Final attempt before throwing - return await client.GetAsync( - documentId, - g => g.Index(client.ElasticsearchClientSettings.DefaultIndex), - cancellationToken); - } - - /// - /// Polls Elasticsearch search until results are found or timeout occurs. - /// - public async Task> WaitForSearchResultsAsync( - Action> configureSearch, - CancellationToken cancellationToken, - TimeSpan? timeout = null, - TimeSpan? pollInterval = null) - { - // 30s overall budget: GitHub-hosted runners are markedly slower than local - // dev machines and the first SearchAsync after index creation can spend - // several seconds priming query caches even after Refresh.True returns. - timeout ??= TimeSpan.FromSeconds(30); - pollInterval ??= TimeSpan.FromMilliseconds(100); - - var client = Services.GetRequiredService(); - using CancellationTokenSource overallCts = new(timeout.Value); - using var overallLinked = - CancellationTokenSource.CreateLinkedTokenSource(overallCts.Token, cancellationToken); - - // Force an index-level refresh up front. SearchIndexService writes documents - // with Refresh.True (`?refresh=true`), which is supposed to guarantee - // immediate searchability — but on slow CI disks the per-document refresh - // is observed to not always propagate before the first SearchAsync. The - // explicit Indices.RefreshAsync here is defensive and idempotent: locally - // it's a no-op (everything's already refreshed), on CI it converts an - // invisible flake into a passing search. - try - { - await client.Indices.RefreshAsync( - r => r.Indices(client.ElasticsearchClientSettings.DefaultIndex), - overallLinked.Token); - } - catch (OperationCanceledException) when (overallLinked.Token.IsCancellationRequested) - { - // Fall through to the final attempt below. - } - - while (!overallLinked.Token.IsCancellationRequested) - { - try - { - var response = await client.SearchAsync(configureSearch, overallLinked.Token); - - if (response.Documents.Count > 0) - { - return response; - } - } - catch (OperationCanceledException) when (overallLinked.Token.IsCancellationRequested) - { - break; - } - - try - { - await Task.Delay(pollInterval.Value, overallLinked.Token); - } - catch (OperationCanceledException) - { - break; - } - } - - // Final attempt with the caller's token only so the assertion sees real - // "found nothing" data rather than a TaskCanceledException at the wait boundary. - return await client.SearchAsync(configureSearch, cancellationToken); - } - - private async Task WaitForElasticsearchAsync() - { - Uri elasticUri = new(ElasticsearchUri + "/"); - using HttpClient http = new() { Timeout = TimeSpan.FromSeconds(2) }; - - for (var i = 0; i < 30; i++) - { - try - { - var response = await http.GetAsync($"{elasticUri}_cluster/health"); - if (response.IsSuccessStatusCode) - { - return; - } - } - catch (HttpRequestException) - { - // Container not ready yet - } - - await Task.Delay(500); - } - - throw new InvalidOperationException("Elasticsearch failed to become ready"); - } -} -``` - -> The two ES polling helpers and `WaitForElasticsearchAsync` are copied verbatim (comments included) from the current `WorkerTestBase.cs`; only the index/uri sources move to base properties. `WaitForDocumentAsync`/`WaitForSearchResultsAsync` remain `public` to satisfy `mustPreserve` (Services consumers call them through the inherited member). `System.Net.Http.HttpClient` and `System.Uri` are available via SDK implicit usings; add `global using System.Net.Http;` to the lib's GlobalUsings if a build error proves otherwise (§9.2). - ---- - -## 3. `Paperless.slnx` (MODIFY) - -Add one `` line (flat — consistent with the existing layout). Order is cosmetic; place after the test projects. - -```xml - - - - - - - - - - - -``` - -> The NUKE `Compile` target builds `GetSolutionPath()` (the whole slnx), so this single line is what makes CI build `Paperless.TestSupport`. No `Pipeline/` change required. - ---- - -## 4. `PaperlessREST.Tests/Integration/SharedRestContainerFixture.cs` (MODIFY) - -> **⚠ Implementation correction (post-validation).** The "remove ALL `Environment.SetEnvironmentVariable` -> and feed config via `ConfigureAppConfiguration` in-memory" plan below **does not work for this WAF -> fixture** and was reverted. `WebApplicationFactory` (minimal hosting) builds the app's own -> configuration, whose environment-variable source — populated process-globally by `.env.test` -> (`TestEnv.Load`) — outranks anything the factory adds via `ConfigureAppConfiguration`; even -> `config.Sources.Clear()` only touches the host-config layer. Result: the REST host bound -> `RABBITMQ__URI=localhost:5672` and **every endpoint 500'd (`BrokerUnreachable`)** — caught by the -> integration suite, not by build/unit tests. The shipped fixture therefore sets the infra **env vars** -> to the Testcontainers endpoints in `ConfigureSutAsync` (the original, proven mechanism — the only -> thing the WAF host reads). The env-var-mutation removal **succeeds for the Services fixture** (a plain -> `Host.CreateApplicationBuilder` where `Configuration.Sources.Clear()` genuinely controls app config); -> it is **not achievable for the minimal-hosting WAF**. Sections 4.1/4.2 below describe the abandoned -> in-memory approach and are kept for the record. - -### 4.1 What changes - -- Class derives from `ContainerFixtureBase` instead of implementing `IAsyncLifetime` directly. Stays `public sealed`. -- `UsesPostgres => true`. -- **Remove the Elasticsearch container entirely** (REST has zero ES consumers — §7). *But the base always starts ES.* See §7 for the chosen resolution: the base starts ES for both; REST simply never queries it. (Alternative — a `UsesElasticsearch` flag — is documented in §9.3 as an open decision; the proportionate default keeps ES in the base since it is harmless and the Services fixture needs it, avoiding a second abstract knob for one consumer.) -- **Remove ALL `Environment.SetEnvironmentVariable(...)` mutation.** Feed `WebApplicationFactory` via `ConfigureAppConfiguration` in-memory (mirroring the Services fixture). This eliminates cross-test global env pollution. -- Static ctor delegates to `TestEnv.Load()`. -- Container fields, bucket creation, and image resolution are gone (now in base). -- `Client`/`Services`/`DbFactory`/`CreateAsyncScope` preserved exactly. `Services` now comes from the base (`protected set`); the fixture assigns `Services = Factory.Services` inside `ConfigureSutAsync`. -- `[assembly: CaptureConsole]`/`[assembly: CaptureTrace]` stay at the top of this file. - -### 4.2 Full intended content - -```csharp -using PaperlessREST.Host; - -[assembly: CaptureConsole] -[assembly: CaptureTrace] - -namespace PaperlessREST.Tests.Integration; - -public sealed class SharedRestContainerFixture : ContainerFixtureBase -{ - static SharedRestContainerFixture() => TestEnv.Load(); - - protected override bool UsesPostgres => true; - - public HttpClient Client { get; private set; } = null!; - public IDbContextFactory DbFactory { get; private set; } = null!; - - public AsyncServiceScope CreateAsyncScope() => Services.CreateAsyncScope(); - - private WebApplicationFactory? _factory; - - protected override async ValueTask ConfigureSutAsync() - { - _factory = new ConfiguredWebApplicationFactory( - PostgresConnectionString, - RabbitConnectionString, - MinioEndpoint, - MinioAccessKey, - MinioSecretKey, - BucketName, - ElasticsearchUri); - - Client = _factory.CreateClient(); - Services = _factory.Services; - DbFactory = Services.GetRequiredService>(); - - await using var db = await DbFactory.CreateDbContextAsync(); - await db.Database.MigrateAsync(); - } - - protected override async ValueTask DisposeSutAsync() - { - if (_factory is not null) - await _factory.DisposeAsync(); - } - - private sealed class ConfiguredWebApplicationFactory( - string postgresConnectionString, - string rabbitConnectionString, - string minioEndpoint, - string minioAccessKey, - string minioSecretKey, - string bucketName, - string elasticsearchUri) - : WebApplicationFactory - { - protected override void ConfigureWebHost(IWebHostBuilder builder) - { - // Replaces the old Environment.SetEnvironmentVariable(...) global mutation. - // WebApplicationFactory reads these via the host's IConfiguration just like - // the Services fixture's AddInMemoryCollection. Colon-keyed to match the - // option binding (ConnectionStrings:*, Storage:Minio:*, Elasticsearch:*). - builder.UseEnvironment("Test"); - builder.ConfigureAppConfiguration((_, config) => - { - config.AddInMemoryCollection(new Dictionary - { - ["ConnectionStrings:PaperlessDb"] = postgresConnectionString, - ["ConnectionStrings:Hangfire"] = postgresConnectionString, - ["RabbitMQ:Uri"] = rabbitConnectionString, - ["Storage:Minio:Endpoint"] = minioEndpoint, - ["Storage:Minio:AccessKey"] = minioAccessKey, - ["Storage:Minio:SecretKey"] = minioSecretKey, - ["Storage:Minio:BucketName"] = bucketName, - ["Storage:Minio:UseSsl"] = "false", - ["Elasticsearch:Uri"] = elasticsearchUri, - // [Required] + ValidateOnStart on ElasticsearchOptions.DefaultIndex → the REST host - // throws at CreateClient() without it. Thread base IndexName through the factory ctor - // (add an `indexName` ctor param alongside the others) so env-var dependence is TRULY - // removed — not silently satisfied by .env.test's ELASTICSEARCH__DEFAULTINDEX process env. - ["Elasticsearch:DefaultIndex"] = indexName - }); - }); - - builder.ConfigureTestServices(services => - { - services.RemoveAll(); - - services.RemoveAll>(); - - var dataSource = new NpgsqlDataSourceBuilder(postgresConnectionString) - .MapEnum("document_status") - .Build(); - - services.AddPooledDbContextFactory(opts => - opts.UseNpgsql(dataSource)); - - services.RemoveAll(); - services.AddSingleton(new MemoryStorage()); - - services.AddFakeLogging(); - }); - } - } -} -``` - -> **Config-key verification required (§9.4):** the old code set process env vars `CONNECTIONSTRINGS__PAPERLESSDB`, `RABBITMQ__URI`, `STORAGE__MINIO__*`, `ELASTICSEARCH__URI` (double-underscore = colon). The in-memory keys above use the colon form of the SAME keys, so the app's existing option binding (`PaperlessREST/Configuration/*` + `Host/Extensions/ServiceCollectionExtensions.cs`) resolves identically. **The implementer MUST grep `PaperlessREST` for the exact config section names** (`GetConnectionString("PaperlessDb")` vs a bound `ConnectionStrings` options class, the `RabbitMQ` section key casing, `Storage:Minio` vs `Storage__Minio`) and match the keys precisely. If any binding is case- or path-sensitive in a way that differs from the documented `appsettings`/options sections, adjust the dictionary keys — do NOT reintroduce `Environment.SetEnvironmentVariable`. `ASPNETCORE_ENVIRONMENT=Test` is replaced by `builder.UseEnvironment("Test")` which is the WAF-native, non-global equivalent. - ---- - -## 5. `PaperlessREST.Tests/Integration/DocumentEndpointTests.cs` (MODIFY, light) - -Only the cleanup mechanism changes — the four `[Fact]`s, their arrange/act/assert, and all assertions are untouched. The manual `_createdDocIds` list + `IAsyncLifetime.DisposeAsync` block is replaced by an `AsyncCleanup` instance created in the constructor and disposed by xUnit. This both demonstrates the RAII helper and removes the bespoke tracking. Behavior is equivalent: the same documents are deleted via the same `ExecuteDeleteAsync`. - -### 5.1 Precise edits - -The class keeps `IClassFixture, IAsyncLifetime` (xUnit invokes `IAsyncLifetime.DisposeAsync`). Replace the fields + lifecycle + helper to route through `AsyncCleanup`. - -- Keep field `private readonly List _createdDocIds = [];` (still the record of what to delete). -- Replace the `IAsyncLifetime` region: - -```csharp -#region IAsyncLifetime - -public ValueTask InitializeAsync() => ValueTask.CompletedTask; - -public ValueTask DisposeAsync() => _cleanup.DisposeAsync(); - -#endregion -``` - -- Add the cleanup field, initialized in the constructor so it captures `_fixture` and `_createdDocIds`: - -```csharp -private readonly AsyncCleanup _cleanup; - -public DocumentEndpointTests(SharedRestContainerFixture fixture) -{ - _fixture = fixture; - _cleanup = new AsyncCleanup(async () => - { - if (_createdDocIds.Count == 0) return; - await using var scope = _fixture.CreateAsyncScope(); - var factory = - scope.ServiceProvider.GetRequiredService>(); - await using var db = await factory.CreateDbContextAsync(); - await db.Documents.Where(d => _createdDocIds.Contains(d.Id)).ExecuteDeleteAsync(); - }); -} -``` - -- `_createdDocIds.Add(...)` / `.Remove(...)` call sites inside the tests are **unchanged** (the list is still the source of truth). - -> This edit is optional-but-recommended (it is the one place the spec spends "new abstraction" budget on a consumer). If the implementer judges it adds risk for little gain, the fallback is: keep `DocumentEndpointTests` exactly as-is and let `AsyncCleanup` be exercised only by being present in the shared lib. **Recommendation: do the edit** — it is behavior-equivalent and is the canonical demonstration the uplift exists. Flagged in §9.5. - ---- - -## 6. `PaperlessREST.Tests/Integration/DatabaseFixture.cs` (MODIFY, light) - -`DatabaseFixture` is NOT migrated to the base (it is Postgres-only with a hand-built DI graph — different shape, out of scope). Two surgical de-duplications only; **public surface unchanged**: - -- Static ctor: `static DatabaseFixture() => TestEnv.Load();` (was `Env.TraversePath().Load(".env.test");`). -- Constructor: `_container = TestContainers.Postgres();` (was the inline `new PostgreSqlBuilder(...).WithWaitStrategy(...).Build()` — the factory produces the identical wait strategy). - -Everything else in `DatabaseFixture` (the `ConfigurationBuilder` with `AddEnvironmentVariables().AddInMemoryCollection`, `MapEnum`, batch options, repositories, `AddFakeLogging`, migration) is left intact. - ---- - -## 7. REST Elasticsearch container — resolution - -**Finding:** No `PaperlessREST.Tests` test resolves `ElasticsearchClient` or queries Elastic. The old `SharedRestContainerFixture` started an ES container and set `ELASTICSEARCH__URI` purely so the app's DI could bind an Elastic client at startup; nothing exercised it. - -**Decision (proportionate):** The base fixture (`ContainerFixtureBase`) always starts ES because the Services fixture genuinely needs it and the REST app's DI binds an `Elasticsearch:Uri` at host build time. Keeping ES in the base for both fixtures: -- preserves REST app startup (DI binds the Elastic client from `Elasticsearch:Uri`), -- avoids a second abstract knob (`UsesElasticsearch`) introduced for a single asymmetric consumer (YAGNI), -- the REST ES container was *already* being started before this refactor, so this is **not** a new cost. - -The CLAUDE.md "dead weight" observation is recorded as an **open decision** (§9.3): a follow-up could add `protected virtual bool UsesElasticsearch => true;` and have REST return `false` *only after* confirming the REST host can boot without an `Elasticsearch:Uri` (today it likely cannot, because the Elastic client is registered unconditionally in REST's DI). That confirmation is a behavior question outside this refactor's no-behavior-change mandate, so ES stays started for REST in this pass. - ---- - -## 8. Test-project edits - -### 8.1 `PaperlessREST.Tests/PaperlessREST.Tests.csproj` (MODIFY) - -Add the project reference next to the existing one: - -```xml - - - - -``` - -Package refs are **kept as-is** (do NOT prune `Testcontainers.*`, `Minio`, `DotNetEnv`, `Microsoft.Extensions.Diagnostics.Testing` from the test project even though TestSupport now also references them). Reason: the test project still uses these types directly outside the fixtures (e.g. `DatabaseFixture` uses `PostgreSqlBuilder`/`NpgsqlDataSourceBuilder`, `FakeLogCollector` is used across many unit tests, `Minio` args in `UploadPdfAsync` paths). Transitive references via TestSupport would technically flow, but explicit refs are clearer and avoid `ImplicitUsings`/analyzer churn. Net new line: one `ProjectReference`. - -### 8.2 `PaperlessServices.Tests/PaperlessServices.Tests.csproj` (MODIFY) - -```xml - - - - -``` - -### 8.3 `GlobalUsings.cs` (both test projects) (MODIFY) - -Add one line each so the moved `FakeLoggerExtensions`, `ContainerFixtureBase`, `AsyncCleanup`, `TestEnv`, `TestContainers`, `MinioBucket` resolve without per-file `using`: - -`PaperlessREST.Tests/GlobalUsings.cs` — append: -```csharp -// Shared test support -global using Paperless.TestSupport; -``` - -`PaperlessServices.Tests/GlobalUsings.cs` — append: -```csharp -// Shared test support -global using Paperless.TestSupport; -``` - -> Both projects already have `global using Microsoft.Extensions.Logging.Testing;` (for `FakeLogCollector`/`FakeLogRecord`) and the Elastic/Testcontainers usings, so the moved members compile at their call sites. - -### 8.4 Delete the duplicated extension files - -- DELETE `PaperlessREST.Tests/FakeLoggerExtensions.cs` -- DELETE `PaperlessServices.Tests/FakeLoggerExtensions.cs` - -All `GetFullLoggerText`/`WaitForLogAsync`/`WaitForLogCountAsync` call sites resolve to `Paperless.TestSupport.FakeLoggerExtensions` via the global using. (15 consumer files total; none change.) `ListenerLifecycleTests.cs`'s *private local* `WaitForLogAsync` is untouched and continues to shadow nothing — it has a different signature and no `this` receiver. - ---- - -## 9. `PaperlessServices.Tests/Integration/WorkerTestBase.cs` (MODIFY) - -### 9.0 What changes - -- `SharedContainerFixture` derives from `ContainerFixtureBase`. Stays `public`. -- `UsesPostgres => false`. -- Containers, image consts, ports, `_bucketName`/`_indexName`/`_host`, `WaitForElasticsearchAsync`, and the two ES polling helpers move to the base — DELETED from this file. -- `Services` comes from the base; the fixture assigns it in `ConfigureSutAsync` (`Services = _host.Services`). -- `UploadPdfAsync(string)` stays here (Services-specific; uses `IMinioClient` from `Services` and the base's `BucketName`). -- Config injection (`AddInMemoryCollection` with `Storage:Minio:*`, `Elasticsearch:*`, `RabbitMQ:Uri`) stays here — uses base properties (`RabbitConnectionString`, `MinioEndpoint`, etc.) instead of local container fields. -- `FakeTextSummarizer` registration stays here. -- `SharedContainerCollection` (`[CollectionDefinition(Name)]`, `const string Name = "SharedContainer"`) is **unchanged** — kept in this file verbatim. -- `[assembly: CaptureConsole]`/`[assembly: CaptureTrace]` stay at the top of this file. - -### 9.1 Full intended content - -```csharp -using PaperlessServices.Host.Extensions; - -[assembly: CaptureConsole] -[assembly: CaptureTrace] - -namespace PaperlessServices.Tests.Integration; - -/// -/// Collection definition for shared container fixture. -/// This ensures containers only start when integration tests run. -/// -[CollectionDefinition(Name)] -public class SharedContainerCollection : ICollectionFixture -{ - public const string Name = "SharedContainer"; -} - -public class SharedContainerFixture : ContainerFixtureBase -{ - static SharedContainerFixture() => TestEnv.Load(); - - protected override bool UsesPostgres => false; - - private IHost _host = null!; - - protected override async ValueTask ConfigureSutAsync() - { - var builder = Microsoft.Extensions.Hosting.Host.CreateApplicationBuilder(); - builder.Configuration.Sources.Clear(); - builder.Configuration.AddInMemoryCollection(new Dictionary - { - ["RabbitMQ:Uri"] = RabbitConnectionString, - ["Storage:Minio:Endpoint"] = MinioEndpoint, - ["Storage:Minio:AccessKey"] = MinioAccessKey, - ["Storage:Minio:SecretKey"] = MinioSecretKey, - ["Storage:Minio:BucketName"] = BucketName, - ["Storage:Minio:UseSsl"] = Environment.GetEnvironmentVariable("MINIO_USE_SSL") ?? "false", - ["Elasticsearch:Uri"] = ElasticsearchUri, - ["Elasticsearch:DefaultIndex"] = IndexName - }); - - builder.Services.AddLogging(b => - { - b.ClearProviders(); - b.AddFakeLogging(o => - { - o.OutputFormatter = r => $" [{r.Level}] {r.Category}: {r.Message}"; - o.OutputSink = Console.WriteLine; - }); - b.SetMinimumLevel(LogLevel.Trace); - }); - - builder.Services.AddPaperlessRabbitMq(builder.Configuration); - builder.Services.AddOcrServices(); - builder.Services.AddSingleton(); - - _host = builder.Build(); - Services = _host.Services; - await _host.StartAsync(); - } - - protected override async ValueTask DisposeSutAsync() - { - // _host is assigned in ConfigureSutAsync. If init throws before that line - // (e.g. a container wait-strategy times out), _host is still null; the base - // guards this call so a naive _host.StopAsync() NRE cannot mask the real - // InitializeAsync exception in xUnit's collection-fixture cleanup report. - if (_host is not null) - { - try { await _host.StopAsync(); } - catch { /* best-effort: don't mask the InitializeAsync exception */ } - _host.Dispose(); - } - } - - public async Task UploadPdfAsync(string content) - { - var fileName = $"test-{Guid.NewGuid():N}.pdf"; - var pdfPath = await Pdf.Create(Dye.White).AddText(content).SaveAsync(fileName); - - var storageKey = $"documents/{TimeProvider.System.GetUtcNow():yyyy-MM}/{Guid.NewGuid():N}/{fileName}"; - var client = Services.GetRequiredService(); - - await using var stream = File.OpenRead(pdfPath); - await client.PutObjectAsync(new PutObjectArgs() - .WithBucket(BucketName) - .WithObject(storageKey) - .WithStreamData(stream) - .WithObjectSize(stream.Length) - .WithContentType("application/pdf")); - - return storageKey; - } -} -``` - -> `WaitForDocumentAsync`/`WaitForSearchResultsAsync` are now inherited from `ContainerFixtureBase` (public), so the four `[Collection]`-bound tests that call `fixture.WaitFor*` compile unchanged. `Services` is inherited (public). `UploadPdfAsync` resolves `IMinioClient` from the Services DI graph (registered by `AddOcrServices`/`AddPaperlessRabbitMq`) using `BucketName` from the base. - ---- - -## 10. Verification plan - -1. `./build.sh Compile` — must stay 0 errors / 0 warnings. This is the floor. Confirms: TestSupport compiles infra-only; both test projects resolve moved members via global usings; no `mustPreserve` member changed signature (a break would surface as a compile error in a consumer). -2. Grep gate (no regressions of the constraints): - - `[assembly: CaptureConsole]` appears exactly twice repo-wide (once per test assembly), never in `Paperless.TestSupport/`. - - `Environment.SetEnvironmentVariable` appears 0 times in `SharedRestContainerFixture.cs`. - - `Env.TraversePath` appears only inside `Paperless.TestSupport/TestEnv.cs`. - - `FakeLoggerExtensions` defined exactly once (in TestSupport). -3. `./build.sh IntegrationTests` under OrbStack (per MEMORY: verify locally, don't wait on CI) — the five integration test classes must pass unchanged. Particularly validates §4.2's config-key migration (if a key is wrong, REST endpoint tests fail at startup/DB connect) and §5's `AsyncCleanup` (documents still deleted). -4. `./build.sh UnitTests` — confirms the moved `GetFullLoggerText`/`WaitForLog*` extensions still resolve for the 15 unit-test consumers and `OcrWorkerTests`. - ---- - -## 11. Out of scope (explicitly NOT changed) - -- `FakeTextSummarizer` (stays internal in Services tests, registered by the Services fixture). -- The Postgres + `DocumentPersistence` + `MapEnum` DAL setup (REST-only, stays in `ConfiguredWebApplicationFactory`). -- `ListenerLifecycleTests.cs` private `WaitForLogAsync` local method. -- `DatabaseFixture` public surface and DI graph (only `.env.test` load + Postgres builder de-duplicated). -- Any `Pipeline/` file (whole-solution build picks up the new project from `Paperless.slnx`). -- Adding/removing/renaming any `[Fact]`/`[Theory]`.