docs(nunit): record Task 8 live verification and remaining P0 gaps

Capture Revit 2023/2026 evidence, Dynamo nunit inventory, async UI deadlock and generation.retained blockers, and link the digest from the agent index.

Author: trgiangv

docs/agents/index.md

@@ -9,7 +9,7 @@ Build/deploy traps: `.agents/skills/build/SKILL.md`.
 |------|------|----------|
 | Compile / deploy / verify | `build` skill, `verification.md`, `build-matrix.md`, `known-test-gaps.md` | `source/`, `scripts/`, `tests/` |
 | MCP integration testing | `mcp-integration-test.md` | Host + daemon |
-| NUnit host testing (experimental) | `nunit-host-testing.md`, `docs/product/nunit-host-testing.md` | `DevTools.NUnit.*` |
+| NUnit host testing (experimental) | `nunit-host-testing.md`, `nunit-native-runtime-verification.md`, `docs/product/nunit-host-testing.md` | `DevTools.NUnit.*` |
 | MCP agent efficiency | `docs/plans/completed/2026-07-26-mcp-agent-efficiency.md` | `DevTools.Mcp.*`, daemon |
 | Execution / MCP / host pipe / logging | Matching `docs/agents/*.md` + `docs/architecture/<Module>/` + `docs/product/` | `DevTools.*`, hosts |
 | Revit API + live execute | `revit-developer` skill, `architecture/MCP/workflows.md` | MCP + rvtdocs-mcp |

docs/agents/nunit-native-runtime-verification.md

@@ -0,0 +1,96 @@
+# NUnit Native Runtime — Live P0 Verification
+
+Evidence for plan Task 8 (`docs/plans/active/2026-08-12-nunit-native-runtime-mtp.md`).
+Raw artifacts: `%LOCALAPPDATA%\RevitDevTool\task8-evidence\`.
+
+## Environment (2026-08-12)
+
+| Host | PID | Build | Notes |
+|------|-----|-------|-------|
+| Revit 2023 | 45108 | 23.1.90.15 | net48; Dynamo cores loaded |
+| Revit 2026 | 43776 | (running) | net8 proxy for plan’s Revit 2025 ALC row (operator decision: same TFM) |
+| Revit 2025 | — | — | Not run; substituted by 2026 |
+| Revit 2027 | — | — | Not installed — environment blocker |
+| AutoCAD | — | — | Not run — optional / blocker |
+
+## Revit 2023 + Dynamo / `nunit.framework`
+
+Dynamo add-in assemblies loaded in-process (13 name matches including `DynamoCore`, `DynamoRevitDS`, …).
+
+**On disk (Dynamo):**
+
+`C:\Program Files\Autodesk\Revit 2023\AddIns\DynamoForRevit\nunit.framework.dll`  
+→ `nunit.framework, Version=2.6.3.13283, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77`
+
+**Loaded in AppDomain after Task1 + gen-two runs:** only DevTools generation copies  
+`nunit.framework, Version=4.6.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb` under  
+`%TEMP%\…\DevTools\NUnit\Generations\<generation_id>\nunit.framework.dll`  
+(count grew with generations; Dynamo’s 2.6.3 identity was **not** loaded into the AppDomain during this session).
+
+Coexistence claim for this run: Dynamo is loaded; DevTools owns its private 4.6 generation `nunit.framework` identities; Dynamo’s older framework stays on disk until a Dynamo test path loads it.
+
+## Task 1 fixture (net48) on Revit 2023
+
+- Discover: **31** cases, `generation_id=cd071c38…982e`.
+- Sync matrix (exclude `AsyncLifecycleFixture` + `FullSemanticsFixture.AsyncTest_Completes`):
+
+| Outcome | Count |
+|---------|-------|
+| Passed | 25 |
+| Skipped (Ignore/Explicit) | 2 |
+| Inconclusive | 1 |
+| Error (deliberate unexpected exception) | 1 |
+
+Summary file: `2023-task1-sync-out.json`. Host PID **45108**.
+
+### Async hang (open gap)
+
+Running `AsyncTest_Completes` / `AsyncLifecycleFixture` alone times out: NUnit `Run` is marshaled onto the Revit UI executor (`ExecuteAsync`), so `WaitForCompletion` blocks Idling while async continuations cannot complete → **UI-thread deadlock**. Full Task1 including async is blocked until Run is off-UI or async fixtures are adjusted.
+
+## Generation-two (same Revit 2023 PID)
+
+Without restarting Revit 45108:
+
+| Assembly | Marker output | `generation_id` |
+|----------|---------------|-----------------|
+| staged `net48-gen1` | (prior sync run) | `cd071c38…982e` |
+| staged `net48-gen2` | `generation-marker=generation-two` | `0e48aa2f…039a` |
+
+File: `2023-gen2-marker-out.json`.
+
+## Revit 2026 (net8 / 2025 proxy)
+
+- Sync Task1 matrix: same outcomes as 2023 (`2026-task1-sync-out.json`).
+- Gen-two marker: `generation-marker=generation-two`, new `generation_id`.
+- ALC unload diagnostic after generation switch: **`generation.retained`**  
+  (`Generation ALC retained after unload verification.`) — observed twice  
+  (`2026-gen2-marker-out.json`, `2026-gen1-again-out.json`).  
+  Generation switch works; clean ALC collect was **not** observed.
+
+## `dotnet test` Revit API smoke (2023)
+
+```powershell
+dotnet test samples/DevTools.NUnit.SampleTests/DevTools.NUnit.SampleTests.csproj `
+  -c Debug.Autodesk.2023 --filter "FullyQualifiedName~Arithmetic_runs_inside_host"
+```
+
+- `Arithmetic_runs_inside_host` **Passed**; stdout includes `23.1.90.15` and `host-pid=45108`.
+- Adapter still executed sibling sample tests in the same run (`Intentional_failure_for_demo` failed, `Writes_output` passed) — filter/adapter gap; acceptance evidence is the Arithmetic case + PID match.
+
+## Cancel-on-disconnect (related hardening)
+
+Earlier same day: Infinite-sleep CancelProbe entered on PID 45108 → concurrent discover blocked → kill Runner → discover completed ~8.8s → MCP execute responsive. See session notes; not re-run in this evidence pack.
+
+## Task 8 close criteria vs this pack
+
+| Plan item | Status |
+|-----------|--------|
+| 2023 Dynamo + `nunit.framework` inventory | Done (Dynamo loaded; 2.6.3 on disk; 4.6 generations in AD) |
+| 2023 Task1 discover/run | Sync matrix done; **async excluded** (deadlock) |
+| 2023 generation-two same PID | Done |
+| 2025 ALC unload | **Proxy 2026**: switch OK, diagnostic **`generation.retained`** |
+| 2027 live | Blocker — not installed |
+| AutoCAD smoke | Blocker / not run |
+| `dotnet test` Revit API smoke | Arithmetic Passed on PID 45108 (filter imperfect) |
+
+**Verdict:** do **not** treat Task 8 as fully closed. P0 live path is largely proven on 2023 + 2026, but async UI deadlock, ALC `generation.retained`, and 2027/AutoCAD blockers remain.

docs/plans/active/2026-08-12-nunit-native-runtime-mtp.md

@@ -1,4 +1,4 @@
-# NUnit Native Runtime And MTP-First Integration Plan
+# NUnit Native Runtime And MTP-First Integration Plan
 
 **Goal:** Run full NUnit 4.6.1 semantics inside Autodesk hosts through a native
 framework runtime, with MTP as the sole IDE/Test Explorer compatibility layer.
@@ -16,7 +16,7 @@ Date: 2026-08-12
 
 ## Status
 
-Accepted — **P0 implementation largely landed in tree; Task 8 live matrix not
+Accepted — **P0 implementation largely landed in tree; Task 8 live matrix not
 fully closed.** See Progress and Remaining Gaps below. Do not start P1 MTP
 work until Task 8 open gaps are resolved or explicitly waived.
 
@@ -225,7 +225,7 @@ public sealed record NUnitRuntimeEvent(
 its reference to `DevTools.NUnit.Core` to the already-loaded host copy so the
 cast remains valid across ALC/no-context boundaries.
 
-## P0 — Correct NUnit Runtime
+## P0 — Correct NUnit Runtime
 
 ### Task 0: Correct assembly ownership before extending the runtime
 
@@ -241,8 +241,7 @@ cast remains valid across ALC/no-context boundaries.
 - Test: `tests/DevTools.NUnit.Core.Tests/`
 - Test: `tests/DevTools.NUnit.Host.Tests/`
 
-- [ ] Add a failing contract-boundary test proving Core references neither
-  `DevTools.Ipc` nor `System.Text.Json`.
+- [x] Add a failing contract-boundary test proving Core references neither`n  ``DevTools.Ipc`` nor ``System.Text.Json``.
 - [ ] Make Core contain only plain DTOs, runtime interfaces, protocol constants,
   and transport-neutral compatibility data.
 - [ ] Keep exactly one loose Core assembly for Host and Runtime type identity.
@@ -313,8 +312,8 @@ and matching PDBs.
 - Modify: `tests/DevTools.NUnit.Core.Tests/NUnitProtocolGoldenTests.cs`
 - Modify: `tests/DevTools.NUnit.Core.Tests/ProtocolCompatibilityTests.cs`
 
-**Produces:** protocol version `2` and the interfaces listed in “Runtime
-Interfaces”.
+**Produces:** protocol version `2` and the interfaces listed in “Runtime
+Interfaces”.
 
 - [x] First add golden JSON tests for traits/properties, source file/line,
   parent test ID, skip reason, attachments, generation ID, and runtime
@@ -618,7 +617,7 @@ P0 live verification does not depend on the later MTP `serve` endpoint.
 - Update progress/evidence in this plan after each run.
 
 Evidence pack (2026-08-12): `docs/agents/nunit-native-runtime-verification.md` +
-`%LOCALAPPDATA%\RevitDevTool\task8-evidence\`. **Not fully closed** — see verdict.
+`%LOCALAPPDATA%\RevitDevTool\task8-evidence\`. **Not fully closed** — see verdict.
 
 - [x] Revit 2023: let Runner locate/reuse or launch Revit with the installed
   Dynamo add-in loaded; do not start Revit directly. Record the selected PID
@@ -628,7 +627,7 @@ Evidence pack (2026-08-12): `docs/agents/nunit-native-runtime-verification.md` +
   Dynamo 2.6.3 on disk under `AddIns\DynamoForRevit`, not loaded this session.)_
 - [~] Revit 2023: discover and run the Task 1 net48 fixture; record expanded
   cases, lifecycle log, result summary, generation manifest, and host PID.
-  _(Discover 31 cases; sync matrix 25/2/1/1 — async fixtures hang on UI executor.)_
+  _(Discover 31 cases; sync matrix 25/2/1/1 — async fixtures hang on UI executor.)_
 - [x] Revit 2023: rebuild to generation two without restarting Revit and prove
   new IL plus generation-specific dependency behavior.
   _(gen2 marker on same PID; new `generation_id`.)_
@@ -645,7 +644,7 @@ Evidence pack (2026-08-12): `docs/agents/nunit-native-runtime-verification.md` +
   logic in the selected Revit PID. The default P0 bridge is the passing smoke
   case in `samples/DevTools.NUnit.SampleTests`; exclude its deliberate demo
   failure from the acceptance invocation.
-  _(Arithmetic Passed, `host-pid=45108`; adapter still ran siblings — filter gap.)_
+  _(Arithmetic Passed, `host-pid=45108`; adapter still ran siblings — filter gap.)_
 - [x] Before each deploy, kill Revit and rebuild the add-in for the target year.
   After deployment, allow only Runner to establish the live session. Reuse that
   session for generation-two proof; do not kill it merely to rebuild a test
@@ -654,7 +653,7 @@ Evidence pack (2026-08-12): `docs/agents/nunit-native-runtime-verification.md` +
 - [ ] Stop P1 if Revit 2023 plus Dynamo or Revit 2025 fails. Choose between a
   documented net48 host-restart requirement and a reduced support matrix;
   never expand reflection emulation.
-  _(2023+Dynamo coexistence OK for this pack; modern ALC unload not clean —
+  _(2023+Dynamo coexistence OK for this pack; modern ALC unload not clean —
   hold P1 on unload/async gaps.)_
 
 Required compile proof follows `.agents/skills/build/SKILL.md`:
@@ -693,13 +692,13 @@ Tracked evidence: [`docs/agents/nunit-native-runtime-verification.md`](../../age
 | Modern ALC reports `generation.retained` | **P0 blocker** | Revit 2026 (net8 proxy for 2025) switches generations but unload verifier never sees collect. Diagnose retain roots; do not treat as unload pass. |
 | Task1 full suite (incl. async) not green live | P0 | Blocked by async hang; sync matrix already recorded. |
 | Adapter/`dotnet test` filter runs sibling sample tests | P1-adjacent | Arithmetic smoke OK on PID; filter still executes demo fail. |
-| Revit 2027 live | Env blocker | Not installed — record only. |
+| Revit 2027 live | Env blocker | Not installed — record only. |
 | AutoCAD smoke | Optional / blocker | Not run. |
 | Cancel-on-disconnect | Done (hardening) | Pipe CT + RuntimeManager CT; live Infinite-sleep probe on 2023 passed. |
 
 **Hold P1** until async executor + ALC unload gaps are closed or the support matrix is explicitly reduced in ADR/plan.
 
-## P1 — MTP-Only IDE Integration
+## P1 — MTP-Only IDE Integration
 
 ### Task 9: Make Runner the explicit host-process owner
 
@@ -750,7 +749,7 @@ public sealed record NUnitRunnerHostInfo(
   existing explicit launch/cancellation policy. Never terminate a reused host.
 - [ ] Add a framed Runner endpoint for hello, discover, run, cancel, and
   shutdown. Forward host progress without converting it to console prose.
-- [ ] Record the package launcher → `Runner serve` boundary in the architecture
+- [ ] Record the package launcher → `Runner serve` boundary in the architecture
   document. Prove that MTP can neither start nor stop the endpoint and that
   endpoint shutdown policy stays with Runner and its external launcher.
 - [ ] Include authoritative Autodesk host PID and ownership in every session
@@ -911,7 +910,7 @@ P1 exit gate: Visual Studio MTP run passes on Revit 2024 and 2025, Rider MTP run
 passes without native-provider hijack, C# Dev Kit MTP run passes on Revit 2025,
 and logs prove MTP delegated process policy to Runner in every case.
 
-## P2 — Packaging And Optional Ergonomics
+## P2 — Packaging And Optional Ergonomics
 
 ### Task 13: Package one consumer surface and retire the emulator
 
@@ -1031,9 +1030,9 @@ and clean sample restore/run evidence.
 - [x] Prioritized execution plan drafted for review.
 - [x] Plan accepted.
 - [x] On acceptance, mark the 2026-08-10 active plan superseded.
-- [x] Tasks 0–7A implemented in tree (Core/Transport, Runtime, generation
+- [x] Tasks 0–7A implemented in tree (Core/Transport, Runtime, generation
   loading, Host manager, Runner CLI v2, packaging ownership).
-- [~] Task 8 live host matrix — partial evidence recorded; **not closed**
+- [~] Task 8 live host matrix — partial evidence recorded; **not closed**
   (async UI deadlock, `generation.retained`, 2027/AutoCAD blockers).
 - [ ] P0 exit gate complete (blocked on Task 8 remaining gaps).
 - [ ] P1 Runner activation/ownership boundary complete.
@@ -1050,7 +1049,7 @@ and clean sample restore/run evidence.
 - Runtime proof: full NUnit semantics fixture, real IL generation change,
   dependency-version change, conflicting NUnit identity, and modern ALC unload.
 - Live proof: Revit 2023 plus Dynamo recorded; Revit 2025 substituted by
-  Revit 2026 (net8) per operator — unload still `generation.retained`.
+  Revit 2026 (net8) per operator — unload still `generation.retained`.
 - Operational live proof: agents use Runner rather than directly starting
   Revit, keep the live lane serialized, record the selected host PID, and kill
   the host before deploying changed add-in binaries.