feat(tools): SDK-level cross-source memory deduplication - #1531
Conversation
Move profile deduplication into the SDK middleware. Facts are normalized (strip leading date, trim, collapse whitespace, casefold) and deduplicated in static > dynamic > search priority within each request, then injected as one owned <supermemory> block that replaces the previous block instead of accumulating. Dedup is mode-aware so query-mode search results are not dropped against an omitted profile. No global/browser Set: request-local only, safe for concurrent requests and Cloudflare Workers. Covers AI SDK, OpenAI Chat/Responses, Mastra, and VoltAgent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Claude finished @Dhravya's task in 2m 2s —— View job Code Review Complete
Result: No issues found — this looks good to ship. ✅ The SDK-level deduplication implementation is solid:
Score: 10/10 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Overview: Moves memory deduplication from the playground UI into the SDK middleware, implementing request-scoped context block replacement across all TypeScript SDK integrations (AI SDK, OpenAI, Mastra, VoltAgent).
Issues found: None — this looks good to ship.
Details:
The implementation is solid:
-
Request isolation is correct — The
seenMemoriesSet indeduplicateMemories()is created fresh per call, and theinjectedflag in the middlewaremap()callbacks is closure-scoped per request. No global state that could cause cross-user leakage. -
Context block regex is safe — The pattern
/<supermemory context="user-memories" readonly>[\s\S]*?<\/supermemory>/uses non-greedy matching with fixed anchors, avoiding ReDoS concerns. The specific attribute match (context="user-memories" readonly) ensures user-authored supermemory tags with different attributes won't be stripped. -
Normalization logic is sound —
normalizeMemoryFact()strips date prefixes, collapses whitespace, and lowercases for comparison while preserving the original memory string for output. The priority order (static > dynamic > search) is maintained correctly. -
Edge cases handled — Empty memories return early without injecting content. Multiple system messages get the context injected into only the first one, with stale blocks stripped from subsequent ones.
-
Good test coverage — Tests verify context replacement (not accumulation), cross-user isolation, and normalized deduplication.
Score: 10/10
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
supermemory-mcp | 2fa2e0d | Aug 18 2026, 03:19 PM |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
supermemory-app | 2fa2e0d | Commit Preview URL Branch Preview URL |
Aug 18 2026, 03:19 PM |
2 participants
Stack Context
This stack moves memory deduplication out of the playground UI and into the SDKs themselves, so every integration injects a single, deduplicated, self-replacing memory block. Three PRs:
sdk-dedup/tools-ts(this PR) — TypeScript SDK core + integrationssdk-dedup/python— Python SDKssdk-dedup/playground— playground debug view reflects the SDK-owned blockWhat?
Move profile deduplication into the SDK middleware for the TypeScript tools package.
[YYYY-MM-DD], trim, collapse whitespace, casefold) and deduplicated instatic > dynamic > searchpriority within a single request.<supermemory>block that replaces the previous block instead of accumulating a new one each turn.Set. Safe for multiple users, concurrent requests, and Cloudflare Worker isolates.Covers AI SDK, OpenAI (Chat + Responses), Mastra, and VoltAgent. New
shared/memory-context.tsowns the block-replacement logic.Why?
The earlier "conversation-scoped deduplication" was only a playground browser
Set— a UI debug affordance that did not change what the SDK sent to the model, and would have been unsafe as server-side global state. Real cross-source dedup belongs in the SDK, applied fresh per stateless model request.Testing
bun run testinpackages/tools: 145 passed (the one failing suite,claude-memory.test.ts, is a pre-existing broken import unrelated to this change).🤖 Generated with Claude Code
Note
Medium Risk
Changes how system prompts and instructions are built across all TypeScript integrations; behavior is well-covered by unit tests but incorrect strip/replace logic could drop or duplicate context in production prompts.
Overview
Moves cross-source memory deduplication and owned prompt injection into
@supermemory/toolsso every integration sends one deduplicated memory block per request instead of growing context each turn.Deduplication: Facts are normalized via
normalizeMemoryFact(strip[YYYY-MM-DD], trim, collapse whitespace, lowercase) and deduplicated with static → dynamic → search priority.deduplicateMemoriesForModekeeps search hits in query mode when the profile is not injected.Owned
<supermemory>block: Newshared/memory-context.tswraps memories in<supermemory context="user-memories" readonly>, strips stale blocks, and replaces prior SDK context while preserving caller system instructions. Applied in AI SDK (injectMemoriesIntoParams), OpenAI Chat/Responses middleware, Mastra input processor (wrapMemoryContext), and VoltAgent hooks.Tests: Unit coverage for block replacement (with-supermemory, OpenAI, VoltAgent), Mastra wrapper tag assertion, normalized dedup variants, and concurrent
containerTagisolation.Reviewed by Cursor Bugbot for commit 2fa2e0d. Bugbot is set up for automated code reviews on this repo. Configure here.