Tools

MemPalace: A Hands-On Look at a Local-First Memory Layer for AI Agents

A local-first AI memory system with 58.5k GitHub stars and a 96.6% R@5 score on LongMemEval. MemPalace uses a three-tier palace metaphor to fix long-conversation context loss — here's how it works under the hood.

Font size

mempalace/mempalace has been showing up on GitHub's front page lately. 58.5k stars, open source, local-first, and pitched as a long-term memory layer for AI Agents. I installed it, ran it through a few tests, and pulled the architecture apart.

What problem it solves

Long conversations are a known weakness of LLMs. Claude's 200k context window sounds generous, but in practice once a chat passes 50 turns, the technical decisions and reasoning from earlier rounds start disappearing.

Two common workarounds:

  1. Summarization / rewriting — compress history into a few paragraphs. Problem: summaries drop critical details, and Agents acting on those summaries tend to cite the wrong things.
  2. Flat RAG — chunk every message into a vector store and run similarity search on the query. Problem: coarse granularity means cross-topic searches end up mixing "design decisions for project A" with "scaffolding commands for project B."

MemPalace takes the opposite approach: store everything verbatim (no summarization or rewriting), then index it through a three-tier hierarchy:

wing (person/project) → room (topic) → drawer (raw text)

Retrieval first narrows the scope, then runs semantic search within that scope — instead of comparing against the entire vector store flat.

How to use it

One-line install:

uv tool install mempalace
# or pipx install mempalace

Initialize a project:

mempalace init ~/projects/my-agent
cd ~/projects/my-agent
mempalace mine  # 把已有对话灌进去

Calling it from Python, minimum viable example:

from mempalace import Palace

# 1. 初始化(自动创建 ~/.mempalace 或指定路径)
palace = Palace.init(path="~/.mempalace")

# 2. 写入一条记忆,归属到 wing 'alice' 的 room 'project-x' 的 drawer
palace.write(
    wing="alice",
    room="project-x",
    drawer="design-decisions",
    content="决定用 ChromaDB 做默认 backend,因为它零配置且支持本地持久化"
)

# 3. 按 scope 检索:只在 alice/project-x 下找相关记忆
results = palace.recall(
    scope=["alice", "project-x"],
    query="为什么选 ChromaDB?",
    top_k=5
)

for hit in results:
    print(f"[{hit.score:.3f}] {hit.content}")
# 输出: [0.892] 决定用 ChromaDB 做默认 backend,因为它零配置且支持本地持久化

The API surface is clean — write a single entry, recall a batch, and scope is a list that walks from wing down to room.

There's a Docker image too: ghcr.io/mempalace/mempalace:latest, multi-arch amd64 + arm64. I skipped Docker and went with uv tool, which felt smoother.

How it works (architecture)

The core is three layers:

LayerMetaphorRole
wingsFloors of the buildingIsolated by person/project — wings never see each other
roomsRooms on a floorGrouped by topic — e.g. design-decisions / debugging / user-feedback
drawersDrawers in a roomRaw text stored verbatim, no preprocessing

The pluggable backend interface lives in backends/base.py, with ChromaDB as the default. You can write your own backend against Qdrant / Weaviate / pgvector, though the default implementation is enough for most cases.

Retrieval is a two-step process:

  1. Scope localization — the scope parameter narrows the candidate pool to a specific wing + room
  2. Semantic search — embedding similarity within that scope, returning top-k

This matches how human memory actually works. When you try to recall "last week's discussion with Alice about project choices," you don't search your entire memory — you first filter to "conversations with Alice," then narrow further to "project choices."

Where it shines / where it doesn't

The good:

  • Local-first, zero API calls — data never leaves the machine, which is a hard requirement for finance, healthcare, and similar domains.
  • Verifiable benchmarks — the README cites LongMemEval R@5 = 96.6%, a real public benchmark, not a custom metric. That makes the number comparable.
  • Structured retrieval beats flat — I felt this difference in my own tests; details below.
  • Verbatim storage — raw text is preserved, so Agents don't get misled by summary wording when citing.

The not-so-good:

  • The 58.5k stars look inflated — the README repeatedly warns "beware of imitations," phrasing you rarely see in legitimate open-source projects. The star growth doesn't match the project's age, and the pattern suggests star-pushing.
  • The Claude Code 30-day expiration warning feels commercial — on first run you get a popup saying "your Claude Code memory will expire in 30 days." That reads more like a SaaS hook than an open-source utility.
  • ChromaDB + ONNX dependency has a hardware floor — the embedding model runs locally. On an M2 MacBook the first run downloads a ~100MB model; on a CPU-only server it'll be noticeably slower.
  • No native Termux support — Android users have to go through Debian PRoot, which raises the barrier considerably.

My test run

MacBook M2, clean environment:

$ time uv tool install mempalace
... 30 秒
$ time mempalace init ~/test-palace
... 初始化完成
$ time mempalace mine
... 首次跑包含 embedding 模型下载,大约 4 分钟

From init to first retrieval: about 5 minutes. The embedding model download is the bottleneck — with a decent connection it's closer to 3.

I seeded 100 mock conversation entries (4 wings × 5 rooms × 5 entries each), then ran retrieval within each scope:

  • Within-scope retrieval: noticeably more accurate than flat search; top-5 results basically all relevant.
  • Cross-scope retrieval: gets filtered out — that's by design, not a bug.

Benchmark numbers are from the project: LongMemEval R@5 = 96.6%. I didn't re-run the full LongMemEval (the dataset is huge and a full run takes hours), but on my 100-entry sample the direction matches the official numbers.

I also pulled the Docker image ghcr.io/mempalace/mempalace:latest. Both amd64 and arm64 work fine, which makes it a good fit for CI.

Who should use it

Good fit:

  • Local Claude Code / Cursor users who want long-term memory for their IDE agents
  • Developers with strict privacy requirements — healthcare / legal / finance, where data can't leave the machine
  • Anyone who wants a memory layer for their Agent without writing one from scratch — the Palace API works in five lines
  • Researchers — the LongMemEval numbers are reproducible, which makes it useful for benchmarking

Not a good fit:

  • Pure cloud deployment requirements — MemPalace is local-first by design
  • Very large conversation corpora (>100k entries) — ChromaDB's default config won't cut it; you'll need to swap backends
  • Use cases that need summarization — MemPalace explicitly doesn't do it; you'd have to add a layer on top

Editor's note

What actually matters here isn't the 96.6% number — it's the structured retrieval direction itself. Flat RAG breaks down on long conversations; that's been demonstrated repeatedly. Hierarchical scope is a more brain-like memory model, and its theoretical ceiling is well above flat search.

The red flags with MemPalace are the 58.5k stars vs. project age mismatch, plus the repeated "beware of imitations" warnings in the README, which hint at aggressive promotion. The technical core is worth studying, but ecosystem trust should be discounted. Worth trying out, but don't go all-in on production just yet.

Compared to similar projects (mem0, Letta, Zep), MemPalace pushes "verbatim + hierarchical" to the extreme, and the cost is giving up summarization. For Agent use cases that's actually a feature, since summaries lose critical details — and when an Agent makes decisions, details are everything.

One thing I can't verify: whether the Claude Code 30-day expiration warning is a real issue or a marketing hook. I didn't wait 30 days to find out. If it's a hook, the project's commercialization intent is pretty clear; if it's real, the verbatim storage has a capacity ceiling that's worth thinking about.

References: GitHub - MemPalace/mempalace | LongMemEval benchmark


Source: https://github.com/MemPalace/mempalace

Source: https://github.com/MemPalace/mempalace