Beyond RAG: How I Designed AI Memory with Obsidian and Claude Code

“We added a vector DB — memory is sorted.” I used to think that too.

When you’re building an AI application, the first wall you hit is: how do I get the AI to remember our information? Most developers land on RAG — Retrieval-Augmented Generation. Chunk the docs, embed them, retrieve relevant chunks per query, inject them into the prompt. I wrote a primer on RAG basics a while back.

But after implementing RAG, a lot of developers notice something odd: retrieval works fine, yet the AI still doesn’t feel like it remembers anything.

It forgets what you discussed in the last session. User preferences don’t accumulate. You have to re-explain context from scratch every time. This isn’t a bug in your RAG implementation — it’s because RAG and memory solve fundamentally different problems.

RAG and AI Memory Are Separate Components

A Zenn article titled “AI Memory vs RAG” laid this out cleanly. The argument is simple:

RAG handles “retrieval of external knowledge.” AI Memory handles “persistence of dynamic state.” They are not interchangeable — they have fundamentally different jobs.

Here’s what that looks like in practice.

RAG operates on relatively static external knowledge: manuals, FAQs, research papers. Its job is: find the relevant chunk from this corpus and surface it.

AI Memory operates on dynamically changing state: user preferences, task progress, past decisions and their rationale. Its job is: this user said X last time and this approach failed before — here’s why.

With only RAG, even a well-built AI is just a “chatbot with search.” To become an agent that develops a real relationship with the user across time, you need a separate layer.

My Implementation

For the past year I’ve been running a daily knowledge-management system built around Obsidian Vault and Claude Code. Looking back at it, I’d inadvertently built exactly this RAG/Memory separation — just without labeling it that way.

The RAG Layer: Obsidian Vault

My Obsidian Vault accumulates external knowledge I’ve ingested:

• LiteratureNote — summaries of papers and articles I’ve read
• Digest — daily news digests (PubMed, RSS, Semantic Scholar)
• Resources — technical notes and reference material

These are largely static knowledge stock — write once, rarely changed. When Claude Code works on a task, it searches and references these notes as needed. Structurally, this is conventional RAG.

The Memory Layer: Claude Code memory

Claude Code has its own memory system — Markdown files saved under .claude/projects/*/memory/. The memory is typed:

• user — role, skills, preferences (“long TypeScript experience, React beginner”)
• feedback — past feedback (“don’t mock the database in tests,” “skip PRs, push directly”)
• project — dynamic project state (“release freeze starts next week,” “auth refactor is a legal requirement”)
• reference — pointers to external resources (“bug tracker is Linear’s INGEST project”)

These are dynamic state that updates with every session. What was learned last session carries forward to the next.

When Both Layers Work Together

The interesting part is when the two layers collaborate.

Take my daily Digest. When Claude Code builds it, the first thing it does is read from memory: what projects am I currently working on? Precision nutrition PoC, AI tutor business, Google Cloud work — with that context loaded, it then triages the latest items from RSS and PubMed.

“This paper is on GLP-1 agonist polyagonist approaches. Related to the precision nutrition PoC — 🔴 must-read.”

Without memory, the output is a flat list of articles. With memory, the system can judge what actually matters to me. That’s the value that RAG alone cannot deliver.

Why Typed Memory Matters

The design that’s had the most practical impact is categorizing memories by type.

Feedback memory is particularly powerful. At some point I told Claude Code: “Don’t mock the database in tests.” My reason: mismatches between the mock and production had broken migrations. Once that feedback is stored in memory, every subsequent session writes tests against the real database — without me repeating myself.

Critically, I store the why alongside the what. Just the rule “don’t mock the DB” would make the system stubbornly avoid mocks even in cases where mocking is clearly appropriate. The rationale lets it reason: “this situation carries no mock/production divergence risk — mocks are fine here.”

Project memory is perishable. “Release freeze next week” is stale in seven days. So I store with absolute dates, not relative ones (“freeze starts 2026-03-05”), and periodically purge expired entries.

What’s Still Unresolved

Honestly: this system still has real weaknesses.

Memory has no portability. I can’t carry Claude Code’s memory into another AI tool. Every time I switch to ChatGPT I start from zero. Ideally, memory would be stored in a model-agnostic format accessible to any AI agent.

Consistency is managed by hand. If memory says “precision nutrition PoC is in phase 1” but the project has actually moved to phase 2, nothing catches that. My current rule is: when reading memory, cross-check against the actual code and files and update if stale. It doesn’t work reliably.

Scaling is an open question. My MEMORY.md index file is under 200 lines today. What it looks like in one year or three is unknown. Some kind of automatic archival by recency and importance will probably be necessary.

What to Actually Do

For AI application memory design, the first call to make is: does this need RAG, or does it need Memory?

RAG alone is sufficient when:

• You’re building a single-turn Q&A bot
• You need document search or summarization
• Tasks are stateless by nature

Memory is required when:

• You’re building an assistant that develops a long-term relationship with a user
• Work spans multiple sessions
• The agent needs to learn from past decisions

In most practical systems, you need both: RAG to retrieve external knowledge, Memory to maintain user context, and the combination to generate personalized responses.

My Obsidian Vault + Claude Code setup is still a work in progress — but it’s a working example of this two-layer separation running in daily use. If you’ve implemented RAG and felt like something was still missing, the next step is worth thinking about: what does your AI actually remember?