Skip to content

Feature/mongodb extension - #2698

Open
LiangshouX wants to merge 6 commits into
agentscope-ai:mainfrom
LiangshouX:feature/mongodb-extension
Open

Feature/mongodb extension#2698
LiangshouX wants to merge 6 commits into
agentscope-ai:mainfrom
LiangshouX:feature/mongodb-extension

Conversation

@LiangshouX

@LiangshouX LiangshouX commented Aug 13, 2026

Copy link
Copy Markdown

AgentScope-Java Version

2.0.3-SNAPSHOT

Description

Background

This PR adds the agentscope-extensions-mongodb module, providing a MongoDB-backed distributed storage backend for AgentScope Java.

Why MongoDB

Agent 的执行信息(对话历史、思维链、工具调用记录等)通常具有以下特点:

  1. 数据体量大:单个 Session 的完整对话历史可能包含数十轮交互,每轮携带大段非结构化文本,数据量往往达到数十 KB 甚至 MB 级别。
  2. 结构灵活:不同 Agent 的状态字段差异较大,Schema 变更频繁,关系型数据库的 rigid schema 会带来大量 ALTER TABLE 操作。
  3. 单字段长度限制:在企业内部(如银行等金融机构),MySQL 等关系型数据库对单个字段的长度通常有严格限制,存储 Agent 运行信息属于架构红线。

MongoDB 的文档模型天然适合这类场景——单个 Session 对应一个 BSON Document,字段长度不受限制,Schema 灵活可变。作者在实际项目 HiveMind 中已验证了这一方案的可行性,使用 MongoDB 存储 Agent 的 Session 信息与对话历史,运行稳定。

Changes

New files — 6 source + 9 test files:

File Description
pom.xml Module build config; depends on agentscope-core, agentscope-harness, mongodb-driver-sync
MongoDistributedStore.java DistributedStore entry point, aggregates all sub-components
state/MongoAgentStateStore.java AgentStateStore implementation; single-document model with CAS optimistic locking
store/MongoBaseStore.java BaseStore workspace KV implementation with namespace compound index
sandbox/MongoSandboxExecutionGuard.java Distributed lock via TTL documents + atomic findOneAndUpdate acquisition
snapshot/MongoSnapshotSpec.java Sandbox snapshot spec
snapshot/MongoRemoteSnapshotClient.java Sandbox snapshot BSON Binary storage with 7-day TTL
Test (9 files) 93 unit tests + 19 contract tests, 112 total, all pass

Modified files — 3 POM registrations:

File Change
agentscope-extensions/pom.xml Add <module>agentscope-extensions-mongodb</module>
agentscope-dependencies-bom/pom.xml Add mongodb-driver.version property + dependencyManagement entry
agentscope-distribution/agentscope-bom/pom.xml Add BOM entry
agentscope-distribution/agentscope-all/pom.xml Add compile/optional dependency

Design Decisions

Decision Rationale
Sync driver (mongodb-driver-sync) Consistent with Redis/MySQL/PostgreSQL extensions
Single-document model One session = one MongoDB document, keys mapped to top-level BSON fields
CAS optimistic locking Atomic compare-and-swap via findOneAndUpdate + _version_{key} field
List append optimization ListHashUtil sampling hash detects changes; pushEach avoids full rewrites
Key validation ^[a-zA-Z_][a-zA-Z0-9_]*$ prevents . and $ in MongoDB field names
TTL index strategy AgentStateStore 30 days, SnapshotClient 7 days, SandboxGuard immediate (0s)
Index upgrade handling Graceful migration: catch IndexOptionsConflict (error 85), drop old index, recreate with new params

Bug Fixes (discovered during testing)

开发过程中发现并修复了 5 个真实 Bug,均由合约测试覆盖:

Bug Severity Description
save() 不递增 version P0 Updates.inc(versionField, 1L) 缺失,导致乐观锁 CAS 全部失效
saveIfVersion(0) 异常类型错误 P0 findOneAndUpdateMongoCommandException 而非 MongoWriteException,导致并发冲突时直接崩溃
saveIfVersion(UNVERSIONED) 反序列化失败 P1 State 是接口无法被 Jackson 实例化,改为只读取 version 字段
TTL=0 数据立即过期 P0 expireAfterSeconds=0 导致 session 数据 60 秒内被 MongoDB TTL 守护进程清除
索引参数冲突启动失败 P1 从旧版本升级时 IndexOptionsConflict (error 85) 导致应用无法启动

How to Test

# Compile (with dependencies)
mvn -pl agentscope-extensions/agentscope-extensions-mongodb -am compile

# Run all tests (unit tests always run; contract tests require local MongoDB)
mvn -pl agentscope-extensions/agentscope-extensions-mongodb test

# Format check
mvn -pl agentscope-extensions/agentscope-extensions-mongodb spotless:check

Testing Summary

详细测试报告见末尾

  • 112 tests total, 0 failures, 0 errors
  • 93 unit tests (Mockito mock) — CI 自动执行
  • 19 contract tests (真实 MongoDB) — 本地执行,CI 通过 Assumptions.abort() 自动跳过
    • MongoBaseStoreContractTest — 6 tests, 覆盖 KV 存储读写语义
    • MongoAgentStateStoreContractTest — 6 tests, 覆盖版本控制与并发安全
    • MongoIndexLifecycleContractTest — 7 tests, 覆盖索引参数与升级迁移

Checklist

  • Code has been formatted with mvn spotless:apply
  • All tests are passing (mvn test — 112 tests, 0 failures)
  • Javadoc comments are complete and follow project conventions
  • Related documentation has been updated (test report, test plan)
  • Code is ready for review

Related Issue

Closes #2636

References

  • Package structure strictly consistent with Redis/PostgreSQL/MySQL extensions
  • Follows existing AgentScope API naming conventions (builder.mongoClient(), builder.databaseName(), etc.)
  • Author's production experience: HiveMind — MongoDB-backed Agent storage in production

详细测试报告

MongoDB Extension 测试报告与设计解读

Module: agentscope-extensions-mongodb
Version: 2.0.3-SNAPSHOT
Date: 2026-08-13
Status: All 112 tests pass (0 failures, 0 errors)


1. 测试全景

本模块的测试分为三个层次,各层职责明确、互不重叠:

┌─────────────────────────────────────────────────────────────────────┐
│  Layer 3: 手动 E2E 验证                                              │  Spring Boot 应用 + 真实 MongoDB
│  (不在自动化测试中,由开发者通过本地工程执行测试验证功能)                │
├─────────────────────────────────────────────────────────────────────┤
│  Layer 2: 合约测试 — 19 个测试                                        │  需真实 MongoDB,CI 自动跳过
│  MongoBaseStoreContractTest          6 tests                        │
│  MongoAgentStateStoreContractTest    6 tests                        │
│  MongoIndexLifecycleContractTest     7 tests                        │
├─────────────────────────────────────────────────────────────────────┤
│  Layer 1: 单元测试 — 93 个测试                                  	     │  Mockito mock,CI 自动执行
│  MongoBaseStoreTest                 13 tests                        │
│  MongoAgentStateStoreTest           31 tests                        │
│  MongoDistributedStoreTest          14 tests                        │
│  MongoSandboxExecutionGuardTest     17 tests                        │
│  MongoRemoteSnapshotClientTest      16 tests                        │
│  MongoSnapshotSpecTest               2 tests                        │
└─────────────────────────────────────────────────────────────────────┘

合计:112 个自动化测试,全部通过。


2. 为什么需要合约测试

2.1 项目已有的合约测试模式

AgentScope 项目对每种存储接口都定义了一套行为合约(Contract),合约测试的核心思想是:

同一组测试用例,用不同的后端实现来跑,确保所有实现行为一致。

项目中有两份权威合约:

合约测试类 所在模块 参考实现 作用
BaseStoreContractTest agentscope-harness InMemoryStore 定义 KV 存储的标准行为
AgentStateStoreVersioningContractTest agentscope-core InMemoryAgentStateStore 定义状态存储 + 乐观锁的标准行为

这些合约测试使用 Java 的 模板方法模式——基类定义测试逻辑,子类通过 override newStore() 注入不同后端。核心模块注释明确要求:

"Extension modules with Docker-backed stores should add module-local tests when feasible."

2.2 为什么 MongoDB 扩展需要独立编写(而非继承)

我们的 MongoDB 合约测试没有继承基类,而是独立编写。原因是:

  1. BaseStoreContractTest 是 package-private class(没有 public 修饰符),位于 io.agentscope.harness.agent.filesystem.remote.store 包中,跨包无法继承
  2. 搜索语义不同——InMemoryStore.search() 使用前缀匹配(search(["a"]) 返回 ["a","b"] 下的条目),但 MongoBaseStore.search() 使用精确命名空间匹配。直接继承会导致搜索测试失败
  3. 独立编写可以加入 Assumptions.abort()——当 MongoDB 不可用时自动跳过,而非报错

独立编写保证了:测试逻辑与核心合约完全对齐,同时适配 MongoDB 的行为差异。


3. 合约测试设计详解

3.1 MongoBaseStoreContractTest — 6 个测试

位置: src/test/java/.../store/MongoBaseStoreContractTest.java

这 6 个测试覆盖了 BaseStore 接口的全部核心语义:

# 测试名称 验证什么 设计原理
1 putGetRoundTrip_versionStartsAtOne 写入后读取,version 从 1 开始 验证最基本的存取链路和初始版本号
2 put_incrementsVersion 每次 put version 递增 1 MongoDB 使用 $inc 原子操作,验证版本自增正确
3 putIfVersion_successAndConflict CAS 成功和冲突检测 这是乐观并发控制的核心——两个线程看到相同 version,只有一个能写成功
4 putIfVersionZero_createIfAbsent expectedVersion=0 表示"仅当不存在时创建" 这是 MemoryConsolidator 中 watermark 写入使用的模式
5 delete_isIdempotent 删除不存在的 key 不抛异常 幂等性——多次删除安全,生产环境不会因重复删除崩溃
6 search_exactNamespaceMatch 精确命名空间匹配(非前缀) 关键差异点——与 InMemoryStore 行为不同,需要明确记录

第 6 个测试(search)是专门为 MongoDB 编写的,它验证了一个重要行为差异:

// MongoDB: search(["s"]) 只返回 ["s"] 下的条目
// InMemoryStore: search(["s"]) 返回 ["s"] 和 ["s","t"] 等子命名空间下的条目
store.put(List.of("s"), "inNs", Map.of("where", "s"));
store.put(List.of("s", "t"), "inChild", Map.of("where", "s/t"));

List<StoreItem> found = store.search(List.of("s"), 100, 0);
// MongoDB: 只返回 "inNs"(1 条)
// InMemoryStore: 返回 "inNs" 和 "inChild"(2 条)

这个差异不影响实际使用,因为 AgentScope 的协调命名空间(["memory", "consolidation"])没有子命名空间,但必须在测试中明确记录。

3.2 MongoAgentStateStoreContractTest — 6 个测试

位置: src/test/java/.../state/MongoAgentStateStoreContractTest.java

这 6 个测试覆盖了 AgentStateStore 的版本控制语义——这是 AgentScope 防止并发写入冲突的核心机制:

# 测试名称 验证什么 设计原理
1 supportsVersioning 返回 true 基本契约声明
2 getVersioned_absent_returnsVersionZero 不存在的 key 返回 version=0 约定:version=0 表示"从未写入"
3 saveIfVersion_createIfAbsent expectedVersion=0 首次创建,重复创建失败 CAS 写入的关键模式:先检查再写入,MongoDB 用 findOneAndUpdate + 条件过滤实现
4 saveIfVersion_unconditionalOverwrite UNVERSIONED 模式无条件覆盖 紧急恢复场景——绕过版本检查强制写入
5 plainSave_bumpsVersion 普通 save 也递增 version 验证 Updates.inc(versionField, 1L) 在 save() 中生效
6 concurrentWriters_onlyOneSucceeds 2 个线程同时写入,只有 1 个成功 并发安全性的终极验证

第 6 个测试(并发写入)是最关键的,它用 CountDownLatch 精确控制两个线程同时竞争:

// 两个线程同时看到 version=1,同时尝试写入
CountDownLatch ready = new CountDownLatch(2);  // 就绪信号
CountDownLatch start = new CountDownLatch(1);  // 发令枪

// 线程 A 和 B 同时执行 saveIfVersion(..., observed=1)
// MongoDB 的 findOneAndUpdate 是原子操作,只有一个线程能匹配到 version=1 并写入
// 另一个线程的 findOneAndUpdate 找不到 version=1 的文档,返回 null → UNVERSIONED

assertEquals(1, successes.get());  // 必须只有 1 个成功
assertEquals(2L, store.getVersioned(...).version());  // version 递增到 2

3.3 MongoIndexLifecycleContractTest — 7 个测试

位置: src/test/java/.../MongoIndexLifecycleContractTest.java

本组测试的由来: 前两类合约测试(读写语义)通过后,我们通过 example 工程在本地真实 MongoDB 上执行了 E2E 验证(启动 Spring Boot 应用 → 发送聊天请求 → 重启应用验证数据持久性)。正是这次本地 E2E 测试发现了两个 P0 级别的索引 Bug——TTL=0 导致 session 数据在 60 秒内被 MongoDB 自动清除、索引参数冲突导致升级后应用无法启动。这两个 Bug 暴露了索引生命周期在自动化测试中的空白,于是我们补上了这 7 个索引测试,将 E2E 中发现的问题固化为自动化回归保护。

为什么需要单独的索引测试:

MongoDB 的索引参数(TTL 值、sparse、unique)不会体现在读写接口的返回值中,因此前两类合约测试完全无法感知。但索引参数错误会导致:

  • TTL=0 → 数据写入后 60 秒内被 MongoDB 自动删除(用户无法察觉,重启后才发现数据丢失)
  • 索引参数冲突 → 应用启动直接崩溃(无法运行)

这类 Bug 是运维层面的致命问题,本地 E2E 测试证明了它们确实会发生,必须有专门的自动化测试覆盖。

# 测试名称 验证什么 设计原理
1 agentStateStore_compoundIndex (user_id, session_id) 复合索引存在 session 查询的性能保障
2 agentStateStore_ttlIndex_30days _updated_at TTL 索引参数是 2592000 秒(30 天) 防止 TTL=0 数据消失——直接覆盖 Bug 4
3 agentStateStore_ttlUpgrade_fromZero 从旧版 TTL=0 索引升级时不抛 IndexOptionsConflict 防止升级崩溃——直接覆盖 Bug 5
4 baseStore_namespaceIndex namespace 单字段索引存在 命名空间查询性能保障
5 baseStore_compoundIndex (namespace, key) 复合索引存在 KV 查询性能保障
6 sandboxGuard_ttlIndex_immediate expiresAt TTL=0(锁立即过期) 锁文档的语义:进程崩溃后锁必须被 MongoDB 自动回收
7 snapshotClient_ttlIndex_7days createdAt TTL=7 天 快照自动清理策略验证

第 3 个测试(升级测试)是最关键的,它分三个阶段模拟真实升级场景:

// Phase 1: 模拟旧代码 — 手动创建 TTL=0 的索引
upgradeDbRef.getCollection(collName).createIndex(
    new Document("_updated_at", 1),
    new IndexOptions().expireAfter(0L, TimeUnit.SECONDS).sparse(true));

// Phase 2: 新代码构造函数执行 ensureIndexes() — 不得抛出 error 85
MongoAgentStateStore.builder()
    .mongoClient(client).databaseName(upgradeDb).collectionName(collName).build();

// Phase 3: 验证索引已被自动修正为 30 天
Document ttlIndex = indexMap(upgradeDb, collName).get("_updated_at_1");
assertEquals(THIRTY_DAYS_SECONDS, ((Number) ttlIndex.get("expireAfterSeconds")).longValue());

4. 测试基础设施设计

4.1 MongoDB 连接与跳过机制

每个合约测试类都使用 Assumptions.abort() 实现 CI 安全:

@BeforeAll
static void connectMongo() {
    try {
        mongoClient = MongoClients.create("mongodb://localhost:27017");
        mongoClient.getDatabase("ping").runCommand(new Document("ping", 1));
    } catch (Exception e) {
        Assumptions.abort("MongoDB not available: " + e.getMessage());
    }
}

为什么用 Assumptions.abort() 而不是 @Disabled

方法 行为 适用场景
@Disabled 永远跳过,需要手动移除注解才能运行 已知不可用的功能
Assumptions.abort() 条件跳过:MongoDB 存在时正常执行,不存在时优雅跳过 环境依赖型测试——CI 无 MongoDB 则跳过,本地有 MongoDB 则执行

JUnit 5 的 Assumptions 机制让同一个测试在不同环境下自动适配,无需维护两套测试配置。

4.2 数据库隔离策略

每个测试类使用带时间戳的独立数据库名

dbName = "test_state_contract_" + System.currentTimeMillis();
// 例如: test_state_contract_1755082549123

为什么不用固定数据库名:

  • 固定名(如 test_db):多个测试并行或测试失败后残留数据会污染后续运行
  • 时间戳名:每次运行唯一,@AfterAlldb.drop() 清理,彻底消除数据残留

4.3 测试生命周期

@BeforeAll    → 连接 MongoDB,创建带时间戳的数据库
  @BeforeEach → 创建新的 store 实例(每次测试隔离)
    @Test     → 执行测试逻辑
  @AfterEach  → 清理当前 session 数据
@AfterAll     → 删除整个数据库,关闭连接

@BeforeEach 每次创建新 store 而不是复用,是为了确保每个测试从干净状态开始,避免测试间相互影响。


5. 测试过程中发现并修复的 Bug

在编写和执行合约测试的过程中,发现了 5 个真实 Bug,这些 Bug 在纯单元测试(Mockito mock)中无法暴露,只有对真实 MongoDB 执行时才会触发:

Bug 1: save() 不递增 version(P0)

现象: plainSave_bumpsVersion 测试失败——期望 version=1,实际 version=0

根因: save() 方法的 MongoDB update 操作只更新了数据字段和 _updated_at,没有 Updates.inc(versionField, 1L)

// 修复前:缺少 version 递增
Bson setFields = Updates.combine(
    Updates.set(key, Document.parse(json)),
    Updates.set(FIELD_UPDATED_AT, new Date()));

// 修复后:加入 version 递增
Bson setFields = Updates.combine(
    Updates.set(key, Document.parse(json)),
    Updates.inc(versionField, 1L),           // ← 新增
    Updates.set(FIELD_UPDATED_AT, new Date()));

影响: 不修复会导致 plain save 后的 version 始终为 0,下游的乐观锁 CAS 全部失效。

Bug 2: saveIfVersion(0) 的 DuplicateKey 异常类型错误(P0)

现象: saveIfVersion_createIfAbsent 测试抛出未捕获的 MongoCommandException

根因: findOneAndUpdate 在 upsert=true 时遇到 DuplicateKey 冲突,抛出的是 MongoCommandException(error code 11000),而不是代码中只捕获的 MongoWriteException

// 修复前:只捕获 MongoWriteException
} catch (MongoWriteException e) {
    if (e.getError().getCode() == 11000) return UNVERSIONED;
    throw e;
}

// 修复后:同时捕获 MongoCommandException
} catch (MongoWriteException e) {
    if (e.getError().getCode() == 11000) return UNVERSIONED;
    throw e;
} catch (MongoCommandException e) {              // ← 新增
    if (e.getErrorCode() == 11000) return UNVERSIONED;
    throw e;
}

影响: 不修复会导致并发场景下 saveIfVersion(0) 直接抛异常崩溃,而不是返回 UNVERSIONED 表示冲突。

Bug 3: saveIfVersion(UNVERSIONED) 反序列化失败(P1)

现象: saveIfVersion_unconditionalOverwrite 测试抛出 JsonException: Failed to deserialize JSON to State

根因: UNVERSIONED 分支调用 getVersioned(... State.class) 来读取写入后的 version,但 State 是接口,Jackson 无法实例化

// 修复前:尝试将数据反序列化为 State 接口
VersionedState<State> vs = getVersioned(userId, sessionId, key, State.class);

// 修复后:只读取 version 字段,跳过数据反序列化
Document doc = collection.find(Filters.eq(slotId))
    .projection(Projections.include(versionField)).first();
Long v = doc.getLong(versionField);
return v != null ? v : 0L;

影响: 不修复会导致所有使用 UNVERSIONED 模式的写入都抛异常。

Bug 4: TTL 索引 expireAfterSeconds=0 导致数据立即过期(P0 — 严重)

现象: 应用运行期间 session 数据正常,重启后 agentscope_sessions 文档内容为空

根因: ensureIndexes() 中 TTL 索引的 expireAfterSeconds 设为 0,意味着 _updated_at 字段一过期文档就立即删除。MongoDB TTL 监控线程每 60 秒扫描一次,会删除所有 _updated_at 已过期的文档

// 修复前(Bug):数据写入后立即可被 TTL 删除
new IndexOptions().expireAfter(0L, TimeUnit.SECONDS).sparse(true)

// 修复后:30 天过期,正常使用中频繁写入会刷新时间戳
new IndexOptions().expireAfter(30L * 24 * 3600, TimeUnit.SECONDS).sparse(true)

影响: 这是用户在实际使用中发现的 Bug——所有 session 数据在 MongoDB TTL 守护进程运行后(最多 60 秒)就会被清除,导致重启后数据丢失。

Bug 5: 索引参数变更时启动失败(P1)

现象: 修复 Bug 4 后,应用启动报 IndexOptionsConflict (error 85)

根因: 数据库中已存在旧的 TTL 索引(expireAfterSeconds=0),代码尝试用新参数(expireAfterSeconds=2592000)创建同名索引,MongoDB 拒绝创建

// 修复后:捕获 error 85,先删旧索引再建新索引
try {
    collection.createIndex(Indexes.ascending(FIELD_UPDATED_AT),
        new IndexOptions().expireAfter(ttlSeconds, TimeUnit.SECONDS).sparse(true));
} catch (MongoCommandException e) {
    if (e.getErrorCode() == 85) {  // IndexOptionsConflict
        collection.dropIndex(ttlIndexName);
        collection.createIndex(...);  // 用新参数重建
    } else {
        throw e;
    }
}

影响: 不修复会导致从旧版本升级时应用无法启动。

Bug 修复总结

Bug 严重性 发现方式 单元测试能发现? 合约测试发现?
save() 不递增 version P0 合约测试 plainSave_bumpsVersion ❌ mock 不检查 MongoDB update 操作
DuplicateKey 异常类型 P0 合约测试 saveIfVersion_createIfAbsent ❌ mock 不模拟 MongoDB 异常类型
UNVERSIONED 反序列化 P1 合约测试 saveIfVersion_unconditionalOverwrite ❌ mock 返回 mock 对象
TTL=0 数据立即过期 P0 用户实际使用发现 ❌ mock 不涉及索引行为 agentStateStore_ttlIndex_30days
索引参数冲突启动失败 P1 用户实际使用发现 ❌ mock 不涉及索引行为 agentStateStore_ttlUpgrade_fromZero

结论: 5 个 Bug 全部能被合约测试覆盖,纯单元测试(Mockito mock)无法发现任何一个。其中 Bug 1-3 由读写语义合约测试直接发现,Bug 4-5 在用户实际使用中首次暴露后,通过新增索引生命周期合约测试补充了自动化回归保护。


6. 测试执行方式

6.1 运行单元测试(CI 自动执行)

# 从项目根目录执行
mvn -f pom.xml -pl agentscope-extensions/agentscope-extensions-mongodb test

输出示例:

Tests run: 112, Failures: 0, Errors: 0, Skipped: 0

6.2 运行合约测试(需要本地 MongoDB)

合约测试包含在上面的命令中。如果本地没有 MongoDB,合约测试会自动跳过(输出中显示 Skipped):

Tests run: 112, Failures: 0, Errors: 0, Skipped: 19   ← 19 个合约测试被跳过

如果本地有 MongoDB(localhost:27017),19 个合约测试会正常执行。

6.3 构建 + 格式检查 + 测试 一体化

# 先格式化代码(Spotless)
mvn -f pom.xml -pl agentscope-extensions/agentscope-extensions-mongodb spotless:apply

# 再构建并测试
mvn -f pom.xml -pl agentscope-extensions/agentscope-extensions-mongodb -am install -DskipTests
mvn -f pom.xml -pl agentscope-extensions/agentscope-extensions-mongodb test

注意: -am 参数(also-make)用于自动构建依赖模块(agentscope-coreagentscope-harness),否则会报依赖找不到。


7. 实际测试执行结果

7.1 执行环境

项目 版本
OS Windows 11
JDK 17
MongoDB 7.x (localhost:27017)
Maven 3.8+
执行日期 2026-08-13

7.2 完整测试结果

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

MongoDistributedStoreTest                    — 14 tests, 0 failures
MongoIndexLifecycleContractTest              —  7 tests, 0 failures  <- 索引生命周期合约测试
MongoSandboxExecutionGuardTest               — 17 tests, 0 failures
MongoRemoteSnapshotClientTest                — 16 tests, 0 failures
MongoSnapshotSpecTest                        —  2 tests, 0 failures
MongoAgentStateStoreContractTest             —  6 tests, 0 failures  <- 合约测试
MongoAgentStateStoreTest                     — 31 tests, 0 failures
MongoBaseStoreContractTest                   —  6 tests, 0 failures  <- 合约测试
MongoBaseStoreTest                           — 13 tests, 0 failures

===============================================
Total: 112 tests | Failures: 0 | Errors: 0 | Skipped: 0
===============================================
BUILD SUCCESS

7.3 合约测试详细结果

BaseStore 合约(6/6 pass):

测试 耗时 状态
putGetRoundTrip_versionStartsAtOne 17ms
put_incrementsVersion 17ms
putIfVersion_successAndConflict 13ms
putIfVersionZero_createIfAbsent 17ms
delete_isIdempotent 10ms
search_exactNamespaceMatch 9ms

AgentStateStore 合约(6/6 pass):

测试 耗时 状态
supportsVersioning 80ms
getVersioned_absent_returnsVersionZero 40ms
saveIfVersion_createIfAbsent 291ms
saveIfVersion_unconditionalOverwrite 21ms
plainSave_bumpsVersion 17ms
concurrentWriters_onlyOneSucceeds 23ms

saveIfVersion_createIfAbsent 耗时较长(291ms)是因为它涉及 findOneAndUpdate 的 upsert 操作,比普通读写多一次 MongoDB 内部的条件检查。

7.3.3 Index Lifecycle 合约(7/7 pass):

测试 耗时 状态
agentStateStore_compoundIndex 119ms
agentStateStore_ttlIndex_30days 9ms
agentStateStore_ttlUpgrade_fromZero 53ms
baseStore_namespaceIndex 367ms
baseStore_compoundIndex 3ms
sandboxGuard_ttlIndex_immediate 19ms
snapshotClient_ttlIndex_7days 17ms

baseStore_namespaceIndex 耗时较长(367ms)是因为首次创建 MongoBaseStore 实例时需要建立连接和创建索引。

测试命令: mvn -f pom.xml -pl agentscope-extensions/agentscope-extensions-mongodb -am test -Dtest="MongoBaseStoreContractTest,MongoAgentStateStoreContractTest,MongoIndexLifecycleContractTest"

image

8. CI 策略与 PR 验收

8.1 CI 中的行为

由于项目未使用 Testcontainers,GitHub CI 无法连接 MongoDB:

测试层次 CI 行为 本地行为
单元测试 (93 个) ✅ 正常执行 ✅ 正常执行
合约测试 (19 个) ⏭️ Assumptions.abort() 跳过 ✅ 正常执行

合约测试的 Assumptions.abort() 机制确保了 CI 不会因为 MongoDB 不可用而报错。

8.2 PR 验收建议

验收项 方式
单元测试 (93 个) CI 自动通过
读写语义合约测试 (12 个) 本地执行,附带终端输出截图
索引生命周期合约测试 (7 个) 本地执行,验证截图见 §7.3.3
E2E 场景 通过搭建 example 测试工程测试验证功能逻辑是否正常,如需提交者(本人)验证所用的工程,我很乐意提供
Bug 修复 查看 git commit 历史,确认 5 个 Bug 均已修复

9. 总结

本次测试工作完成了:

  1. 19 个合约测试——覆盖 BaseStoreAgentStateStore 的读写语义(12 个)+ 全组件索引生命周期(7 个)
  2. 发现并修复 5 个真实 Bug——全部能被合约测试覆盖,纯单元测试无法发现任何一个
  3. 112 个测试全部通过——单元测试 + 合约测试,0 failures,0 errors
  4. CI 兼容——合约测试在 CI 中自动跳过,不影响流水线

合约测试的价值在于:它用真实 MongoDB 驱动代码,暴露了 Mockito mock 无法发现的问题。特别是 findOneAndUpdate 的异常类型差异(MongoCommandException vs MongoWriteException)和 $inc 版本递增行为,只有在真实数据库上才能验证。

…simplify CAS

- Remove duplicate ascending index on createdAt in MongoRemoteSnapshotClient; the TTL index already provides the same sorting capability

- Add Objects.requireNonNull guards to MongoBaseStore constructor for database and collectionName parameters

- Simplify MongoBaseStore.putIfVersion() return: findOneAndUpdate with version filter already guarantees the result matches expectedVersion + 1
…n MongoAgentStateStore

- save() now increments version on each call (Updates.inc)
- saveIfVersion(UNVERSIONED) reads version directly from MongoDB document
  instead of deserializing through State.class interface
- saveIfVersion(0) catches MongoCommandException in addition to
  MongoWriteException for DuplicateKey errors from findOneAndUpdate
- TTL index changed from expireAfterSeconds=0 to 30 days to prevent
  silent data loss by MongoDB TTL monitor
- ensureIndexes gracefully handles IndexOptionsConflict (error 85)
  when upgrading from old TTL index parameters

test(extensions-mongodb): add contract tests for BaseStore, AgentStateStore and index lifecycle

- MongoBaseStoreContractTest (6 tests): CRUD, CAS, idempotent delete, search
- MongoAgentStateStoreContractTest (6 tests): versioning, CAS concurrency
- MongoIndexLifecycleContractTest (7 tests): index parameters, TTL values,
  upgrade from old TTL=0 without IndexOptionsConflict
… safety

Assumptions.abort() in @BeforeAll does not prevent @afterall from running.
When MongoDB is unreachable, @afterall calls client.getDatabase(dbName).drop()
which blocks for 60s then throws MongoTimeoutException — causing CI failure.

Fix: add `connected` flag, set only on successful ping. @afterall guards all
MongoDB operations behind `if (connected)`. Also move dbName assignment before
the try-catch to avoid null in disconnect.

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR adds a new agentscope-extensions-mongodb module providing MongoDB-backed distributed storage for AgentScope. It implements 6 source files covering all DistributedStore components: MongoAgentStateStore (session state with single-document model and CAS optimistic locking), MongoBaseStore (workspace KV store), MongoSandboxExecutionGuard (distributed lock via TTL documents), MongoRemoteSnapshotClient (sandbox snapshot storage), and MongoDistributedStore (facade). The code is well-structured, follows existing extension patterns (Redis/MySQL/PostgreSQL), and includes comprehensive test coverage (112 tests across 9 test files). The overall quality is high — the CAS logic is correct, the single-document model is well-designed, and edge cases (duplicate key errors, TTL index conflicts) are properly handled.

public long saveIfVersion(
String userId, String sessionId, String key, State value, long expectedVersion) {
validateKey(key);
if (expectedVersion == UNVERSIONED) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] Non-atomic read-after-write in saveIfVersion(UNVERSIONED) path

When expectedVersion == UNVERSIONED, save() atomically increments the version via $inc, then a separate find() reads it back. Between these two operations, another concurrent writer could modify the version, causing the returned value to be stale.

While the caller explicitly opted out of versioning semantics (UNVERSIONED), returning a misleading version could confuse downstream logic that uses it for subsequent CAS calls.

Consider combining the write and version read into a single findOneAndUpdate with returnDocument(AFTER).

}

@Override
public void delete(String userId, String sessionId, String key) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] delete(userId, sessionId, key) does not update FIELD_UPDATED_AT

When a specific key is deleted from a session document via $unset, the updatedAt field is not refreshed. Since the TTL index (expireAfter(30 days)) relies on updatedAt, a session where all keys are individually deleted (rather than the whole document being deleted) will have its TTL countdown based on the last save() call, not the delete operation.

Fix: include Updates.set(FIELD_UPDATED_AT, new Date()) in the update.

Indexes.ascending(FIELD_USER_ID), Indexes.ascending(FIELD_SESSION_ID)));

String ttlIndexName = FIELD_UPDATED_AT + "_1";
long ttlSeconds = 30L * 24 * 3600;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] TTL index expiration hardcoded to 30 days

The TTL index on updatedAt is set to 30L * 24 * 3600 seconds (30 days). This is a reasonable default but should be configurable via the Builder to accommodate different session lifecycle requirements (e.g., short-lived test sessions vs. long-lived production sessions).

Similarly, MongoRemoteSnapshotClient hardcodes a 7-day TTL. Consider adding builder parameters.

new FindOneAndUpdateOptions().returnDocument(ReturnDocument.AFTER));

if (reclaimed != null) {
log.debug(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] No lock renewal mechanism — TOCTOU window after TTL expiry

The lock acquisition uses insertOne + ownership verification find. If the TTL expires between these two operations, another process could acquire the lock while the first process believes it still holds it. This is the same fundamental limitation as the Redis SET NX PX approach and is documented as a "safety valve".

However, unlike the Redis implementation (which uses a Lua CAS script for safe release), the MongoDB release() uses deleteOne with owner filter — which is correct but doesn't protect against the TOCTOU window after successful acquisition.

Consider:

  1. Adding a heartbeat/renewal mechanism for long-running operations
  2. Documenting the maximum safe operation duration relative to TTL more prominently in the class Javadoc
  3. Logging a warning in release() if the delete count is 0 (indicating the lock was already reclaimed by another process)


private static final Logger log = LoggerFactory.getLogger(MongoBaseStore.class);

private static final String FIELD_ID = "_id";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Null byte (\0) separator in itemDocId

The compound _id uses \0 as a namespace/key separator. While MongoDB handles null bytes in strings correctly, this can cause issues when exporting data to tools/formats that don't handle null bytes (CSV, some JSON parsers) or debugging via mongosh where null bytes may not display.

The Redis extension uses : as separator. Consider using a visible separator like : or / with proper escaping, or document this design choice explicitly.

MongoSandboxExecutionGuard.builder(mongoClient)
.databaseName(databaseName)
.build();
cachedExecutionGuard = result;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] close() does not propagate to cached components

When close() is called and ownsClient == true, the MongoClient is closed. However, cached MongoAgentStateStore instances (created via agentStateStore()) also implement AutoCloseable but are not closed.

Since they share the same MongoClient (supplied via builder.mongoClient()), their close() won't attempt to close the client (ownsClient=false in their context). So this is functionally safe, but for consistency and future-proofing, consider closing cached components.

private static final String DEFAULT_DATABASE_NAME = "agentscope";
private static final String DEFAULT_COLLECTION_NAME = "agentscope_sessions";
private static final String ANON_USER = "__anon__";
private static final String LIST_SUFFIX = ":list";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] validateKey regex restricts keys to [a-zA-Z0-9_.\-]+ — not documented in AgentStateStore contract

The key validation pattern is a reasonable security measure to prevent BSON field name injection. However, this constraint is not documented in the AgentStateStore interface contract, which could lead to surprises for callers using keys with other characters (e.g., colons, slashes).

Consider documenting this in the class Javadoc and/or proposing a standardized key format constraint at the interface level.

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR adds a new agentscope-extensions-mongodb module providing MongoDB-backed distributed storage for AgentScope. It implements 6 source files covering all DistributedStore components: MongoAgentStateStore (session state with single-document model and CAS optimistic locking), MongoBaseStore (workspace KV store), MongoSandboxExecutionGuard (distributed lock via TTL documents), MongoRemoteSnapshotClient (sandbox snapshot storage), and MongoDistributedStore (facade). The code is well-structured, follows existing extension patterns (Redis/MySQL/PostgreSQL), and includes comprehensive test coverage (112 tests across 9 test files). The overall quality is high — the CAS logic is correct, the single-document model is well-designed, and edge cases (duplicate key errors, TTL index conflicts) are properly handled.

public long saveIfVersion(
String userId, String sessionId, String key, State value, long expectedVersion) {
validateKey(key);
if (expectedVersion == UNVERSIONED) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] Non-atomic read-after-write in saveIfVersion(UNVERSIONED) path

When expectedVersion == UNVERSIONED, save() atomically increments the version via $inc, then a separate find() reads it back. Between these two operations, another concurrent writer could modify the version, causing the returned value to be stale.

While the caller explicitly opted out of versioning semantics (UNVERSIONED), returning a misleading version could confuse downstream logic that uses it for subsequent CAS calls.

Consider combining the write and version read into a single findOneAndUpdate with returnDocument(AFTER).

}

@Override
public void delete(String userId, String sessionId, String key) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] delete(userId, sessionId, key) does not update FIELD_UPDATED_AT

When a specific key is deleted from a session document via $unset, the updatedAt field is not refreshed. Since the TTL index (expireAfter(30 days)) relies on updatedAt, a session where all keys are individually deleted (rather than the whole document being deleted) will have its TTL countdown based on the last save() call, not the delete operation.

Fix: include Updates.set(FIELD_UPDATED_AT, new Date()) in the update.

Indexes.ascending(FIELD_USER_ID), Indexes.ascending(FIELD_SESSION_ID)));

String ttlIndexName = FIELD_UPDATED_AT + "_1";
long ttlSeconds = 30L * 24 * 3600;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] TTL index expiration hardcoded to 30 days

The TTL index on updatedAt is set to 30L * 24 * 3600 seconds (30 days). This is a reasonable default but should be configurable via the Builder to accommodate different session lifecycle requirements (e.g., short-lived test sessions vs. long-lived production sessions).

Similarly, MongoRemoteSnapshotClient hardcodes a 7-day TTL. Consider adding builder parameters.

new FindOneAndUpdateOptions().returnDocument(ReturnDocument.AFTER));

if (reclaimed != null) {
log.debug(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[recommended] No lock renewal mechanism — TOCTOU window after TTL expiry

The lock acquisition uses insertOne + ownership verification find. If the TTL expires between these two operations, another process could acquire the lock while the first process believes it still holds it. This is the same fundamental limitation as the Redis SET NX PX approach and is documented as a "safety valve".

However, unlike the Redis implementation (which uses a Lua CAS script for safe release), the MongoDB release() uses deleteOne with owner filter — which is correct but doesn't protect against the TOCTOU window after successful acquisition.

Consider:

  1. Adding a heartbeat/renewal mechanism for long-running operations
  2. Documenting the maximum safe operation duration relative to TTL more prominently in the class Javadoc
  3. Logging a warning in release() if the delete count is 0 (indicating the lock was already reclaimed by another process)


private static final Logger log = LoggerFactory.getLogger(MongoBaseStore.class);

private static final String FIELD_ID = "_id";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Null byte (\0) separator in itemDocId

The compound _id uses \0 as a namespace/key separator. While MongoDB handles null bytes in strings correctly, this can cause issues when exporting data to tools/formats that don't handle null bytes (CSV, some JSON parsers) or debugging via mongosh where null bytes may not display.

The Redis extension uses : as separator. Consider using a visible separator like : or / with proper escaping, or document this design choice explicitly.

MongoSandboxExecutionGuard.builder(mongoClient)
.databaseName(databaseName)
.build();
cachedExecutionGuard = result;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] close() does not propagate to cached components

When close() is called and ownsClient == true, the MongoClient is closed. However, cached MongoAgentStateStore instances (created via agentStateStore()) also implement AutoCloseable but are not closed.

Since they share the same MongoClient (supplied via builder.mongoClient()), their close() won't attempt to close the client (ownsClient=false in their context). So this is functionally safe, but for consistency and future-proofing, consider closing cached components.

private static final String DEFAULT_DATABASE_NAME = "agentscope";
private static final String DEFAULT_COLLECTION_NAME = "agentscope_sessions";
private static final String ANON_USER = "__anon__";
private static final String LIST_SUFFIX = ":list";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] validateKey regex restricts keys to [a-zA-Z0-9_.\-]+ — not documented in AgentStateStore contract

The key validation pattern is a reasonable security measure to prevent BSON field name injection. However, this constraint is not documented in the AgentStateStore interface contract, which could lead to surprises for callers using keys with other characters (e.g., colons, slashes).

Consider documenting this in the class Javadoc and/or proposing a standardized key format constraint at the interface level.

@AgentScopeJavaBot AgentScopeJavaBot added enhancement New feature or request area/extensions agentscope-extensions (general) labels Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/extensions agentscope-extensions (general) enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: add MongoDB storage extension

2 participants