Memory City Pipeline

Local-first, layered memory for AI agents — stop losing context between sessions, without handing your data to a cloud service you don't control.

Status: Beta / reference implementation — maintained as-is, issues welcome, no SLA.

In plain terms: Memory City treats an AI assistant's memory like a small city — every kind of information has its own district. Active thinking lives in your own notes app (Obsidian works well). Finished, source-backed knowledge lives in wiki pages kept under version control — that's where GitHub fits. Small stable facts the assistant should always remember ("durable memory", handled by Mem0) live in a tiny local database. Repeatable how-tos become "skills". This repo is the routing rules plus the plumbing and safety checks that keep those districts honest — it is not a chatbot, and not a memory engine itself.

Built and battle-tested on a real agent operation. Every component runs on a modest machine (tested on a 2012-era i5 with 12GB RAM), costs nothing to run, and keeps your data in files and databases you own.

Try it in 60 seconds (no install)

The core scripts are stdlib-only and ship with an offline test suite that builds its own fixtures — no pip install, no Docker, no network:

git clone https://github.com/ComicCoder23/memory-city.git
cd memory-city
python tests/test_pipeline.py
# → All tests passed. (11 checks: read-only guarantee, WAL visibility,
#   honest exit codes, stale-FTS detection)

Component status

Component	Status
archive_search.py	Works — proven by the offline test suite
memory_healthcheck.py	Works — proven by the offline test suite
mem0_qdrant_smoke.py	Works — verified end-to-end 2026-07-13 against a live Qdrant + Mem0 stack (store → recall → cleanup, exit 0). Requires the optional dependency stack below.
docker-compose.yml	Works — standard single-container Qdrant

The problem

Agents forget. Every new session starts cold, re-derives context, and repeats mistakes. Cloud memory services fix this by taking your data; this project fixes it by giving your agent a routing discipline plus a small set of local tools.

The architecture

Memory is a routing problem, not a storage problem. Each layer does exactly one job:

Layer	Job	Backed by
Durable memory	Compact, stable facts for automatic recall	Mem0 OSS + Qdrant (local Docker)
Legacy archive	Read-only historical fact store, searched before promoting old facts	SQLite (FTS5)
Session archive wiki	What happened: decisions, actions, outcomes	Markdown / JSON files
Study wiki	Source-backed knowledge with provenance	Markdown files (git)
Working notes	Active thinking, drafts, plans	Any markdown vault (e.g. Obsidian)
Skills	Repeatable procedures	Prompt/script files

Promotion ladder: raw capture → working note → distilled study note → stable procedure. Facts only enter durable memory when they are short, stable, and worth automatic recall. See docs/ROUTING_MATRIX.md.

Memory lanes: facts are tagged by lane (e.g. cat:user_pref, cat:project, cat:general, plus per-person lanes). Lanes keep one person's context from polluting another's, and keep sensitive material out of shared recall. Lane isolation is enforced at write time, not hoped for at read time.

What's in the box

• scripts/archive_search.py — provably read-only search over legacy SQLite fact stores (FTS5 ranked with LIKE fallback). Opens with mode=ro; never checkpoints your WAL.
• scripts/memory_healthcheck.py — honest healthcheck with real exit codes. An unreachable vector store is CRITICAL, never silently "healthy". Checks archive integrity, FTS sync, lane distribution, WAL growth, Qdrant collection status.
• scripts/mem0_qdrant_smoke.py — end-to-end write→recall→cleanup smoke test for the Mem0+Qdrant stack. Cleans up after itself (the naive version leaves a duplicate test fact behind on every run — ask us how we know).
• docker-compose.yml — single-container local Qdrant.
• docs/ — the routing matrix and architecture notes.
• tests/ — offline test suite (builds its own SQLite fixtures; no network, no Docker needed).

Quick start

# 1. Vector store
docker compose up -d

# 2. Python deps
pip install -r requirements.txt

# 3. Smoke test the stack (uses a free Groq key for fact extraction,
#    or a local Ollama model — both optional for infer=False writes)
export GROQ_API_KEY=...   # optional
python scripts/mem0_qdrant_smoke.py

# 4. Health check (exit 0 = healthy, 1 = degraded, 2 = critical)
python scripts/memory_healthcheck.py

# 5. Search a legacy archive read-only
python scripts/archive_search.py "docker" --root ./memory/state/memory_store.db

All paths and endpoints are environment-driven — see the header of each script.

Hard-won lessons baked in

1. sqlite3.connect() is not read-only. A plain connection checkpoints the WAL on close and mutates the db file — dangerous against a live agent's store. Use file:...?mode=ro URIs (still reads the WAL, so recent facts are included).
2. Healthchecks must be able to fail. A check that only tests file existence reports "healthy" while your vector store is down. Exit codes must reflect reality.
3. Smoke tests must clean up. Test writes without deletes pollute recall with junk facts.
4. Don't bulk-import old memory. Search the legacy archive on demand; promote facts one by one only when still true.
5. Lane separation is non-negotiable when multiple people or projects share one agent.

Requirements

• Docker (for Qdrant) — Docker Desktop works fine on Windows
• Python 3.10+
• Optional: a free Groq API key (OpenAI-compatible) or local Ollama for Mem0 fact extraction

Dependency weight warning: the core scripts (archive_search.py, memory_healthcheck.py) and the test suite are stdlib-only — nothing to install. pip install -r requirements.txt is only needed for the Mem0 smoke test, and it pulls PyTorch transitively via sentence-transformers (~2 GB download). Skip it unless you're running the Mem0 path.

Limitations

• archive_search.py and memory_healthcheck.py assume a facts-shaped SQLite store (a facts table with an FTS5 companion). If your schema differs, adapt the table/column names in the script headers — they're deliberately small enough to read in one sitting.
• No memory engine is included; this is ops tooling around your engine (Mem0 or similar), not a replacement for it.
• No schema migration, multi-collection management, or service/cron packaging.
• Developed on Windows 10 + Python 3.10; the stdlib core should run anywhere with 3.10+, but macOS is untested.

Security

No SLA, no dedicated security contact at this scale — report anything concerning via GitHub issues and it will be looked at on a best-effort basis. The scripts make no outbound network calls except the healthcheck's Qdrant ping and the optional Mem0/Groq path.

Contributing — the whole point

This is free and stays free. What it wants is not your money but your collisions: run it against your own memory stack, find where it breaks, and tell me — or better, send the fix. Stars help other people find it; forks and PRs make it evolve. See CONTRIBUTING.md for the two rules and the 60-second test loop.

Fuel the build (entirely optional)

If it saved you an afternoon and you're the tipping kind:

Tier	Damage	What it does
🥤 A can of Irn-Bru	£1.50	Scotland's other national debugging fluid
☕ A proper coffee	£5	Converts directly into one bug hunt
🥩 A steak	£50	A feature ships well-fed

Tip jar: ko-fi.com/comiccoder23 — one-offs welcome, any of the tiers above or your own number. The free tip is still the best one: ⭐ the repo, break it, and file what you broke.

License

MIT — see LICENSE.