fix(harness): make memory flush fire-and-forget to unblock conversation completion

[chcodex]

Summary

• MemoryFlushMiddleware used concatWith to chain the memory flush after the agent stream, so the downstream onComplete was delayed by a full LLM extraction round-trip plus disk writes on every call (default FlushMode.ALWAYS). Switched to doOnComplete + subscribeOn(boundedElastic()).subscribe() so the flush runs in the background and the conversation stream completes immediately.
• Snapshot the conversation list (new ArrayList<>(state.getContext())) before the async flush to avoid concurrent modification if the next agent call clears the state while the flush is still running.
• Applied the same fire-and-forget pattern to MemoryMaintenanceMiddleware, which had the identical concatWith structure.
• Added MemoryFlushMiddlewareAsyncFlushTest to verify the agent stream completes before a never-finishing flush does, and that the flush is still triggered.

Test plan

• New MemoryFlushMiddlewareAsyncFlushTest passes (verifies stream completes before flush + flush still triggered).
• Existing MemoryFlushMiddlewareTriggerTest (17 tests) passes.
• Existing MemoryMaintenanceMiddlewareStaleEntryTest (3 tests) passes.
• Full agentscope-harness module test suite passes (831 tests, 0 failures).

Closes #2774

[chcodex]

Update: graceful shutdown for the async flush (commit 5d99f005)

The first version made the flush fire-and-forget but did not handle shutdown: HarnessAgent.close() could release the workspace while the background flush was still writing memory/*.md, racing with @TempDir deletion in tests (intermittent Failed to delete temp directory under a full test run).

What changed

• New MemoryBackgroundTasks (io.agentscope.harness.agent.memory): process-wide in-flight counter with begin()/end() and awaitQuiescence(timeout, unit). The middleware instances that dispatch the flush are created per agent call, so they are not reachable from close() — the counter is process-wide for the same reason SessionTree.awaitMirrorQuiescence already is.
• MemoryFlushMiddleware / MemoryMaintenanceMiddleware: the async task chain now wraps doOnSubscribe(begin()) / doFinally(end()) so the counter is always paired (success, error, and cancel all release it).
• HarnessAgent.close(): waits for memory-task quiescence (5s) before releasing resources, mirroring the existing SessionTree.awaitMirrorQuiescence call directly above it.
• MemoryFlushMiddlewareAsyncFlushTest: previously used Flux.never(), which would hold the in-flight counter open forever and force every subsequent close() to wait the full 5s timeout. The test now gates the flush stream behind a latch, asserts the agent stream completes while the flush is in-flight, then releases the gate and asserts the tracker quiesces back to zero.

Why this is safe

• The non-blocking behavior from the original fix is preserved: the conversation stream still completes immediately; close() is the only place that waits, and only for tasks that were already running.
• The 5s cap means a genuinely stuck flush cannot hang shutdown indefinitely (same timeout as the session-tree drain).

Verification

• LocalFilesystemPersonalAssistantExampleTest — 10 consecutive full runs, all green (was ~50% flaky before).
• Full agentscope-harness suite — 831 tests, 0 failures.

[codecov]

Codecov Report

❌ Patch coverage is 76.59574% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...pe/harness/agent/memory/MemoryBackgroundTasks.java	75.00%	5 Missing and 2 partials ⚠️
...arness/agent/middleware/MemoryFlushMiddleware.java	77.77%	2 Missing ⚠️
.../agent/middleware/MemoryMaintenanceMiddleware.java	80.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!