Earlier this year, a few good friends and I floated the idea of going to CES.
As time went on, we started thinking... Wait. Spending a few days in Vegas talking about the latest consumer technology? Why haven't we been doing this every year?
I had one thing I especially wanted to see: wearable technology.
We wandered the show floor looking at robots, gaming hardware, AI gadgets, and all kinds of interesting tech. Eventually we found a section where several Chinese manufacturers were showing off display glasses. They had built the hardware, they just needed people to figure out what to do with it.
After trying them out, we all had the same dream.
Jarvis.
A personal AI assistant that's always there. Something that can answer questions, remember your schedule, surface useful information, and maybe even give you a pep talk when you're having a rough day.
Now, don't get me wrong, we're not there yet.
The hardware is still a little clunky, battery life isn't ideal, and the software has a long way to go.
But I have absolutely no doubt this is where we're headed.
And as builders, the beginning is always the most exciting place to be.
So we decided to build our own version of Jarvis.
(We quickly realized we'd need a different name. Disney's lawyers are probably not people you want emailing you.)
As everyone settled into their roles, I kept asking myself one question:
What's the hardest problem to solve?
For me, the answer was obvious.
Memory.
If you've spent much time working with LLMs, you know they're incredible.
They can reason, write, explain, and solve problems that felt like science fiction only a few years ago.
But by themselves, they don't retain long-term memory.
Instead, every time you send a message, the application has to provide the model with the information it needs to understand what's happening. That information is called context.
Most AI chat applications do this by sending previous messages, summaries, and other relevant information along with your latest prompt.
This works surprisingly well...
...until it doesn't.
LLMs are surprisingly human in this regard. Give them too much information and they begin to lose track of what actually matters.
That's why every model has what's called a context window. That's the maximum amount of information it can effectively process at once.
As I sat with this problem, trying to make conversations with AI feel more like conversations with a friend, I started thinking about how human memory actually works.
When something reminds you of your wife, you don't consciously search your brain for "everything related to Kalie."
Instead...
Something happens.
A thought.
A smell.
A conversation.
A restaurant.
And suddenly memories surface almost automatically.
They aren't searched for.
They're triggered.
That became the key idea.
What if, instead of storing memories and searching through them directly, I stored the reasons those memories might become relevant?
Take a simple memory:
"My wife likes sushi."
That memory could have dozens of possible triggers:
- Date night ideas
- My wife's favorite food
- Restaurants
- Things that make Kalie happy
- Anniversary planning
- Trying a new place to eat
Using vector embeddings, a way of representing meaning mathematically so that concepts like coffee and espresso are naturally close together, I could search these triggers instead of searching the raw memory itself.
Now imagine someone says:
"I want to take my wife on a date."
The system doesn't need the exact words "my wife likes sushi."
Instead, it finds related triggers and surfaces that memory automatically.
Now the AI can naturally respond:
"I remember Kalie likes sushi. Here are a few great sushi spots nearby."
That's when the idea really clicked.
Instead of isolated memories, I now had a graph.
Triggers connected to memories.
Memories connected to people.
People connected to places.
Places connected to conversations.
Every connection carried a strength, and every memory accumulated signals over time. Memories that were recalled often became stronger. New experiences reinforced existing connections. Forgotten memories slowly faded into the background.
Now the graph wasn't just storing information, it was learning which information mattered.
Traditional graph algorithms suddenly became useful again.
PageRank could help determine the global importance of a memory based on how many meaningful connections pointed to it.
Edge weights could represent how strongly two ideas were related.
Community detection could identify clusters of related memories.
Graph traversals could explore nearby concepts before deciding what to retrieve.
Vector search became the way to activate the graph, finding semantically related triggers, while the graph itself determined which memories deserved to surface.
Instead of asking:
"Find me everything about my wife."
The system could naturally think:
"We're talking about date night... the strongest connected memories are sushi, that new restaurant she wanted to try, and your anniversary coming up."
That felt much closer to how my own memory works.
Not perfect.
But closer.
A month later, NeuralGraph was born.
← Back to all posts