engineering writeup
Giving a stateless AI agent a memory
The problem
LLM sessions are stateless — every conversation starts cold. I wanted an agent that remembers across sessions and machines, and can search what it knows.
The shape
MCP write → Postgres (source of truth) → single-writer exporter → git / Markdown mirror → local ingest → FTS5 + pgvector index → MCP search back into the agent
The hard parts
Source of truth vs. mirror. Postgres is authoritative; the Markdown files are a generated read-mirror. A single-writer exporter eliminated the multi-session write race that plagued the file-first design.
Hybrid retrieval. FTS5 handles exact/keyword recall; pgvector handles semantic similarity. Results are merged and ranked so a query hits both "the literal string" and "the thing I meant."
Consistency. Snapshots use SQLite's online-backup API rather than the CLI — mixing SQLite builds mid-checkpoint produced torn uploads and one real corruption incident. The online-backup path is consistent even under concurrent writes.
Tiered memory. A lean L1 index points to topic docs, which point to an L4 archive — a "minimum sufficient pointer" discipline that keeps the always-loaded context small.
What I'd do differently
The single-writer exporter fixed the write race for topic docs, but the changelog is still on the old file-plus-git path — it's the one piece of this system with a different sync story than everything else, and that asymmetry is a tax on every session close. It should move onto the same Postgres-source-of-truth model as the rest.
Retrieval is a naive merge of FTS5 keyword hits and pgvector semantic hits today, not a real fusion. A proper reciprocal-rank-fusion or learned re-ranker would converge the two result sets better than interleaving them by raw rank.
The read-mirror only refreshes at session boundaries — each machine pulls the regenerated Markdown and rebuilds its local search index when a session starts, not continuously. A session that starts moments after another machine's export can still serve slightly stale reads for a few minutes.
There's no automated pass to catch near-duplicate memories. The same fact gets re-saved with slightly different wording across sessions, and nothing merges or collapses those; a periodic semantic-similarity sweep over the topic docs would keep the corpus tighter.