A Context Engine
for AI

The most obvious approach: make the context window bigger. GPT-4 Turbo has 128K tokens. Gemini 1.5 claims 1M+. Just stuff all the history in there, right?

Except it doesn't work. Research consistently shows that LLMs degrade on long contexts — the "lost in the middle" problem means information in the center of long prompts is effectively ignored. At 100K+ tokens, models miss relevant details while latency and cost balloon. You're paying to send tokens the model can't use.

And even if the attention problem were solved, context windows are fundamentally the wrong abstraction. You don't want to send everything — you want to send the right things. A 1M-token window with 50 conversations stuffed in is worse than a 4K window with exactly the three facts that matter.

Limitation: Degrades with length, no selectivity, high cost, no persistence across sessions.

RAG is the industry's current default answer to memory: chunk your data, embed it, store it in a vector database, and retrieve the top-K most similar chunks at query time. It's a real improvement over stuffing context, and it works for document Q&A.

But RAG has deep structural limitations when used as a memory system. It stores flat text chunks with no relationships between them. It retrieves based on surface-level semantic similarity — so "I'm stressed about work" might pull up a chunk about your job description instead of the fact that your project deadline is tomorrow. There's no understanding of how pieces of knowledge relate to each other.

RAG also can't adapt. The same query always returns the same chunks (assuming the same embeddings). There's no feedback loop, no learning from what was actually relevant. And because it treats everything as isolated chunks, it can't do things like: "find everything related to this person's work life" — because it doesn't know that your job, your team, your project, and your stress are all connected.

Limitation: No structure, no relationships, no adaptation, retrieval based on text similarity alone.

Some systems use an LLM to summarize past conversations into a running summary that gets prepended to each new conversation. ChatGPT's "Memory" feature works roughly this way — it extracts bullet-point facts from conversations and injects them as system prompt context.

The problem is information loss. Summarization is inherently lossy — the LLM decides what's "important" in a past conversation, and that decision is irreversible. Nuance disappears. If you mentioned a preference in passing three weeks ago, the summarizer probably dropped it. If two facts interact in a subtle way, the summary captured them as isolated bullets.

Worse, summaries are opaque. You can't query them. You can't traverse relationships. You can't ask "what do I know about this user's health?" because the summary is just a flat string. And summaries don't adapt — they represent a fixed snapshot of what the LLM thought was important at summarization time, not what's actually relevant to the current query.

Limitation: Irreversible information loss, no queryability, no structure, no relevance adaptation.

Traditional knowledge graphs (Neo4j, entity-relation triples) preserve structure beautifully — entities have types, relationships have labels, and you can traverse connections. Some AI memory systems like Mem0 build memory graphs that store extracted entities and relationships.

But traditional KGs have a retrieval problem. They require structured queries — you need to know what you're looking for to find it. "Find all nodes connected to David" works. "Find context relevant to this ambiguous natural language message" doesn't. Most KG-backed memory systems fall back to keyword matching or a basic vector layer on top, which negates much of the structural benefit.

They also lack adaptivity. A node has the same importance whether it's been relevant in every conversation or hasn't surfaced in months. There's no feedback, no decay, no learning. And most KG approaches are single-domain — they can store one type of knowledge, but can't compose a personal memory graph with a domain knowledge base at query time.

Limitation: Rigid retrieval, no natural language matching, no adaptation, single-domain.

The newest wave of tools — Zep, Mem0, LangMem — recognize the problem and are building purpose-built memory layers for LLMs. They're a real step forward. Most combine some form of entity extraction with vector storage, and some add graph relationships.

But they still make fundamental tradeoffs. Most rely heavily on LLM calls at retrieval time — meaning memory lookup adds latency and cost to every query. Their retrieval is typically single-channel (vector search, or graph lookup, but not both fused together). They lack the trigger concept — meaning retrieval is always reactive (match what the user said) rather than proactive (surface what the graph knows is relevant based on learned patterns).

Most critically, they're typically single-space systems. You get one memory store per user. You can't compose a personal memory graph with a company knowledge base, an AI personality configuration, and a domain-specific ontology — all in one query, all ranked together. That composability is essential for building real applications.

Limitation: LLM-dependent retrieval, single-channel, no triggers, no multi-space composition.