I'm failing to grasp how it solves/replaces what vector db were created for in the first place (high-dimensional neighborhood searching, where the space to be searched grows by distance^dimension)
It doesn't replace vector db, it's more for storing agentic memory, think of information which you would like agents to remember across conversations with users just like humans
Super simplistic example, but say i mention my Daughter, who is 9.
Then mention she is 10,
a few years later she is 12 but now i call her by her name.
I have struggled to get any of the RAG approaches to handle this effectively. It is also 3 entries, but 2 of them are no longer useful, they are nothing but noise in the system.
> I have struggled to get any of the RAG approaches to handle this effectively.
You need to annotate your text chunks. For example you can use a LLM to look over the chunks and their dates and generate metadata like summary or entities. When you run embedding the combination data+metadata will work better than data alone.
The problem with RAG is that it only sees the surface level, for example "10+10" will not embed close to "20" because RAG does not execute the meaning of the text, it only represents the surface form. Thus using LLM to extract that meaning prior to embedding is a good move.
Make the implicit explicit. Circulate information across chunks prior to embedding. Treat text like code, embed <text inputs + LLM outputs> not text alone. The LLM is how you "execute" text to get its implicit meaning.
(memory) [2024-05-30] My daughter turned 11 today!
System prompt: Today is 2025-08-21
User prompt: How old is my daughter?
The vector DB does the work of fetching the daughter-age-related memories, your system decides (perhaps with another LLM) if the question needs time-based sorting or something else.
Yeah good call, I missed that. I don't think there's a correct answer here, but it could be another step of the read or write. Either it would do another lookup of "my daughter" -> Name on read, or do a lookup on write if you already have a "my daughter is Name" memory. Whatever's less expensive in the long run. The graph memory someone else mentioned also seems like a good option there.
When the RAG sees retrieved facts, it should see the timestamp for each. It will easily then use the latest fact if there aren't too many conflicting ones. I am assuming that the relevance ordering of the retrieved facts won't help.
Separating a RAG from a memory system, it is important for a memory system to be able to consolidate facts. Any decent memory system will have this feature. In our brain we even have an eight hour sleep window where memory consolidation can happen based on simulated queries via dreams.
That is because basic RAG is not very useful as a long-term knowledge base. You have to actively annotate and transform data for it to become useful knowledge. I have the same problem in the regulation domain, which also constantly evolves.
In your case, you do not want to store the age as fact without context. Better is e.g. to transform the relative fact (age) into an absolute fact (year of birth), or contextualize it enough to transform it into more absolutes (age 10 in 2025.
The submission and the readme fail to explain the important thing, which is how BM25 is run. If it creates bags of words for every document for every query, that would be ineffective. If it reuses the BM25 index, it is not clear when it is constricted, updated, and how it is stored.
Because BM25 ostensibly relies on word matching, there is no way it will extend to concept matching.
Interesting! Text files in git can work for small sizes, like your 100MB.
That is what's known in FAISS as a "flat" index, just one thing after another. And obviously you can query by primary key to the key-value store that is git, and do atomic updates as you'd expect. In SQL land this is an unindexed column, you can do primary key lookups on the table, or you can look through every row in order to find what you want.
If you don't need fast query times, this could work great! You could also use SQL (maybe an AWS Aurora Postgres/MySQL table?) and stuff the fact and its embedding into a table, and get declarative relational queries (find me the closest 10 statements users A-J have made to embedding [0.1, 0.2, -0.1, ...] within the past day). Lots of SQL databases are getting embedding search (Postgres, sqlite, and more) so that will allow your embedding search to happen in a few milliseconds instead of a few seconds.
It could be worth sketching out how to use SQLite for your application, instead of using files on disk: SQLite was designed to be a better alternative to opening a file (what happens if power goes out while you are writing a file? what happens if you want to update two people's records, and not get caught mid-update by another web app process?) and is very well supported by many language ecosystems.
Then, to take full advantage of vector embedding engines: what happens if my embedding is 1024 dimensions and each one is a 32 bit floating point value? Do I need to save all of that precision? Is 16-bit okay? 8-bit floats? What about reducing the dimensionality? Is it good enough accuracy and recall if I represent each dimension with an index to a palette of the best 256 floats for that dimension? What about representing each pair of dimensions with an index to a palette of the best 256 pairs of floats for those two dimensions? What about, instead of looking through every embedding one by one, we know that people talk about one of three different topics, and we have three different indices for each of those major topics, and to find your nearest neighbors you want to first find your closest topic (or maybe closest two topics?) and then search in those lower indices? Each of these hypotheticals is literally a different “index string” in an embedding search called FAISS, and could easily be thousands of lines of code if you did it yourself.
It’s definitely a good learning experience to implement your own embedding database atop git! Especially if you run it in production! 100MB is small enough that anything reasonable is going to be fast.
You could use BM25S [0] instead of rank-bm25 for a nice speedup.
Also, there are tradeoffs associated with using BM25 instead of embedding similarity. You're essentially trading semantic understanding for computational speed and keyword matching.
I recently wrote a short anecdote in a similar vein- in my testing, “agentic” retrieval where you simply pass an annotated list of files to an LLM and ask it which ones it wants to look at is probably better than traditional RAG for small datasets (few hundreds of docs).
I found it was both much simpler and more accurate at the cost of marginally more time and tokens, compared to RAG on embedded chunks with a vector store.
I was daunted by the amount of components that goes into a basic RAG. I also ended up starting with "agentic retrieval", which I invented. And it seems that many others have invented same/similar thing. It is just so much easier to start with something simple and improve later on.
Exactly, If it’s dumb and it works it isn’t dumb. (Just have to measure that it does work).
I think this is one of those cases where doing the simplest possible thing is suprisingly good and getting better as LLMs get better. I thought it would be a placeholder with all kinds of problems I would have to hurry to replace but it was surprisingly hard to beat on our most important metric of accuracy. And accuracy got better for “free” when the underlying model got better.
Using a vector db that’s created as a post commithook is probably a reasonable improvement. With context lengths getting bigger natural language is getting easier to work with but your leaving a bunch of gains on the floor without them.
No shade on your project, this is an emerging space and we can all use novel approaches.
Great idea! How does it compare to Graphiti / Zep? They use real time knowledge-graphs to tackle the problem you are trying to solve https://github.com/getzep/graphiti
I think BM25 is the right path for most use cases. I'd reach for a single lucene index here, making the commit hash one of the document's indexed fields. This would make FTS across history nearly instant. I'd store it in an ignored folder right next to .git.
For code search only, BM25 might be a bit overkill and not exactly what you want. FM indexes would be a simpler and faster way to implement pure substring search.
Maybe having both kinds of search at the same time could work better than either in isolation. You could frame them as "semantic" and "exact" search from the perspective of the LLM tool calls. The prompt could then say things like "for searching the codebase use FunctionA, for searching requirements or issues, use FunctionB."
I'm no ai expert so consider my opinion having little value besides casual user.
But I really like this. It seems clever and easy for me to grasp some pros for this approach.
I could envision a bunch of use cases about this workikg well. Ive personally encounter scenadios where sometimes the ai gets hung up on irrelevant outdated fact. But could still look up if specifically needed.
I could see even an automated short summary of all history that is outdated being updated in the vector db from this too. So not all context is lost.
I think part of the reason i like Claude Code so much for non-technical organizational work, is that it's like a controllable memory with human in the loop (as well as a powerful general agent in the loop). Most of what i do is basically just doc creation and management. "Create this doc using this context and store it in this directory." In ChatGPT, I don't have much control or visibility over the memory.
Seems really interesting! It would be great to see some evals to understand how well this works when stacked up against different approaches and use cases.
Keen to do this as well! But couldn't find a good reliable eval for memory creation and retrieval, if you have one you could recommend i'd give it a go asap.
I'm also working on something in this space ("git-native memory for AI assistants"), with a completely different approach though. Will have my assistant check out your repo!
Interesting idea, I like the fact that it doesn't use embeddings, and going into historical memory is an extra step that doesn't impact the speed of resolving the current information
I mean these are two completely different approaches for completely different purposes. So saying that git replaces vector database is like saying that that a filing cabinet replaces a search engines.
This is cool! Do you have any plans on releasing it as an MCP server? Would love to add this to my mcp gateway service https://mcp-boss.com/ (shameless plug)
Would have to be productionized, the current repo is a PoC, but if there is enough interest we could def commit some resources to hardening this and adding things like MCP.
I'm failing to grasp how it solves/replaces what vector db were created for in the first place (high-dimensional neighborhood searching, where the space to be searched grows by distance^dimension)
It doesn't replace vector db, it's more for storing agentic memory, think of information which you would like agents to remember across conversations with users just like humans
Super simplistic example, but say i mention my Daughter, who is 9.
Then mention she is 10,
a few years later she is 12 but now i call her by her name.
I have struggled to get any of the RAG approaches to handle this effectively. It is also 3 entries, but 2 of them are no longer useful, they are nothing but noise in the system.
> I have struggled to get any of the RAG approaches to handle this effectively.
You need to annotate your text chunks. For example you can use a LLM to look over the chunks and their dates and generate metadata like summary or entities. When you run embedding the combination data+metadata will work better than data alone.
The problem with RAG is that it only sees the surface level, for example "10+10" will not embed close to "20" because RAG does not execute the meaning of the text, it only represents the surface form. Thus using LLM to extract that meaning prior to embedding is a good move.
Make the implicit explicit. Circulate information across chunks prior to embedding. Treat text like code, embed <text inputs + LLM outputs> not text alone. The LLM is how you "execute" text to get its implicit meaning.
Hmm, I'm trying to contextualize your insight with the example that was given.
That approach sounds great for a lot of usecases, but wouldn't it still struggle with the given example of the age changing over the years?
How old is x? -> 10
Two years later:
How old is x? -> 12
As a simplified example:
(memory) [2023-07-01] My daughter is 10
(memory) [2024-05-30] My daughter turned 11 today!
System prompt: Today is 2025-08-21
User prompt: How old is my daughter?
The vector DB does the work of fetching the daughter-age-related memories, your system decides (perhaps with another LLM) if the question needs time-based sorting or something else.
Still new to the rag space, but there op had an additional callout, "and [later] i call her by her name"
Is it capable to go from `[name] -> [my daughter as a concept] -> age` ?
Yeah good call, I missed that. I don't think there's a correct answer here, but it could be another step of the read or write. Either it would do another lookup of "my daughter" -> Name on read, or do a lookup on write if you already have a "my daughter is Name" memory. Whatever's less expensive in the long run. The graph memory someone else mentioned also seems like a good option there.
The agent loop takes care of that
When the RAG sees retrieved facts, it should see the timestamp for each. It will easily then use the latest fact if there aren't too many conflicting ones. I am assuming that the relevance ordering of the retrieved facts won't help.
Separating a RAG from a memory system, it is important for a memory system to be able to consolidate facts. Any decent memory system will have this feature. In our brain we even have an eight hour sleep window where memory consolidation can happen based on simulated queries via dreams.
That is because basic RAG is not very useful as a long-term knowledge base. You have to actively annotate and transform data for it to become useful knowledge. I have the same problem in the regulation domain, which also constantly evolves.
In your case, you do not want to store the age as fact without context. Better is e.g. to transform the relative fact (age) into an absolute fact (year of birth), or contextualize it enough to transform it into more absolutes (age 10 in 2025.
I could be wrong, but this seems to be exactly what Zep’s “Graphiti” library is made for, build as a semantic temporal knowledge graph.
I do not necessarily think it is noise, similar to how not all history is noise.
That’s a pretty good analogy though, noise is just information that isn’t immediately useful for the current task
If I need to know the current age I don’t need to know the past ages of someone
Exactly. If you're not using embeddings, why would you need a vector db?
> Why are we building complex vector stores
Because we want to use embeddings.
The submission and the readme fail to explain the important thing, which is how BM25 is run. If it creates bags of words for every document for every query, that would be ineffective. If it reuses the BM25 index, it is not clear when it is constricted, updated, and how it is stored.
Because BM25 ostensibly relies on word matching, there is no way it will extend to concept matching.
Somehow i feel like the op doesn't get what vector DB's are actually for, semantic search isn't the same as a markdown memory file
Interesting! Text files in git can work for small sizes, like your 100MB.
That is what's known in FAISS as a "flat" index, just one thing after another. And obviously you can query by primary key to the key-value store that is git, and do atomic updates as you'd expect. In SQL land this is an unindexed column, you can do primary key lookups on the table, or you can look through every row in order to find what you want.
If you don't need fast query times, this could work great! You could also use SQL (maybe an AWS Aurora Postgres/MySQL table?) and stuff the fact and its embedding into a table, and get declarative relational queries (find me the closest 10 statements users A-J have made to embedding [0.1, 0.2, -0.1, ...] within the past day). Lots of SQL databases are getting embedding search (Postgres, sqlite, and more) so that will allow your embedding search to happen in a few milliseconds instead of a few seconds.
It could be worth sketching out how to use SQLite for your application, instead of using files on disk: SQLite was designed to be a better alternative to opening a file (what happens if power goes out while you are writing a file? what happens if you want to update two people's records, and not get caught mid-update by another web app process?) and is very well supported by many language ecosystems.
Then, to take full advantage of vector embedding engines: what happens if my embedding is 1024 dimensions and each one is a 32 bit floating point value? Do I need to save all of that precision? Is 16-bit okay? 8-bit floats? What about reducing the dimensionality? Is it good enough accuracy and recall if I represent each dimension with an index to a palette of the best 256 floats for that dimension? What about representing each pair of dimensions with an index to a palette of the best 256 pairs of floats for those two dimensions? What about, instead of looking through every embedding one by one, we know that people talk about one of three different topics, and we have three different indices for each of those major topics, and to find your nearest neighbors you want to first find your closest topic (or maybe closest two topics?) and then search in those lower indices? Each of these hypotheticals is literally a different “index string” in an embedding search called FAISS, and could easily be thousands of lines of code if you did it yourself.
It’s definitely a good learning experience to implement your own embedding database atop git! Especially if you run it in production! 100MB is small enough that anything reasonable is going to be fast.
Very interesting idea! I also replaced vector databases with "Table of Content" for retrieval: https://colab.research.google.com/github/VectifyAI/PageIndex...
You could use BM25S [0] instead of rank-bm25 for a nice speedup.
Also, there are tradeoffs associated with using BM25 instead of embedding similarity. You're essentially trading semantic understanding for computational speed and keyword matching.
[0] https://github.com/xhluca/bm25s
I recently wrote a short anecdote in a similar vein- in my testing, “agentic” retrieval where you simply pass an annotated list of files to an LLM and ask it which ones it wants to look at is probably better than traditional RAG for small datasets (few hundreds of docs).
I found it was both much simpler and more accurate at the cost of marginally more time and tokens, compared to RAG on embedded chunks with a vector store.
Shameless plug- https://www.matthewnewton.com/blog/replacing-rag
I was daunted by the amount of components that goes into a basic RAG. I also ended up starting with "agentic retrieval", which I invented. And it seems that many others have invented same/similar thing. It is just so much easier to start with something simple and improve later on.
Exactly, If it’s dumb and it works it isn’t dumb. (Just have to measure that it does work).
I think this is one of those cases where doing the simplest possible thing is suprisingly good and getting better as LLMs get better. I thought it would be a placeholder with all kinds of problems I would have to hurry to replace but it was surprisingly hard to beat on our most important metric of accuracy. And accuracy got better for “free” when the underlying model got better.
Hey Alex, we actually had a very similar idea this morning! I could not find your contact anywhere, would love to chat. This is really cool.
Sure! www.calendly.com/growth-kinetics
Using a vector db that’s created as a post commithook is probably a reasonable improvement. With context lengths getting bigger natural language is getting easier to work with but your leaving a bunch of gains on the floor without them.
No shade on your project, this is an emerging space and we can all use novel approaches.
Keep it up!
ooo nice post-commit creation of the db for the "now" state and links to the files for diff/historical? best of both worlds.
Great idea! How does it compare to Graphiti / Zep? They use real time knowledge-graphs to tackle the problem you are trying to solve https://github.com/getzep/graphiti
I think BM25 is the right path for most use cases. I'd reach for a single lucene index here, making the commit hash one of the document's indexed fields. This would make FTS across history nearly instant. I'd store it in an ignored folder right next to .git.
For code search only, BM25 might be a bit overkill and not exactly what you want. FM indexes would be a simpler and faster way to implement pure substring search.
Maybe having both kinds of search at the same time could work better than either in isolation. You could frame them as "semantic" and "exact" search from the perspective of the LLM tool calls. The prompt could then say things like "for searching the codebase use FunctionA, for searching requirements or issues, use FunctionB."
I'm no ai expert so consider my opinion having little value besides casual user. But I really like this. It seems clever and easy for me to grasp some pros for this approach.
I could envision a bunch of use cases about this workikg well. Ive personally encounter scenadios where sometimes the ai gets hung up on irrelevant outdated fact. But could still look up if specifically needed.
I could see even an automated short summary of all history that is outdated being updated in the vector db from this too. So not all context is lost.
Keep up the great work!
Hey this is awesome! I built something very similar, context-llemur, it’s a CLI and MCP interface to manage context
https://github.com/jerpint/context-llemur
Major difference is a conversation doesn’t get stored, the LLM (or you) can use the MCP/CLI to update with the relevant context updates
I think part of the reason i like Claude Code so much for non-technical organizational work, is that it's like a controllable memory with human in the loop (as well as a powerful general agent in the loop). Most of what i do is basically just doc creation and management. "Create this doc using this context and store it in this directory." In ChatGPT, I don't have much control or visibility over the memory.
Seems really interesting! It would be great to see some evals to understand how well this works when stacked up against different approaches and use cases.
Keen to do this as well! But couldn't find a good reliable eval for memory creation and retrieval, if you have one you could recommend i'd give it a go asap.
Very nice. We tackled the temporal issue "versioned" in your case with a temporal knowledge graph: https://www.falkordb.com/blog/building-temporal-knowledge-gr...
I'm also working on something in this space ("git-native memory for AI assistants"), with a completely different approach though. Will have my assistant check out your repo!
Interesting idea, I like the fact that it doesn't use embeddings, and going into historical memory is an extra step that doesn't impact the speed of resolving the current information
exactly!
And you can _choose_ to explore the history, which in the most common case is not even needed.
Just had a look at the repo -- I'm curious to know where this goes when you add vectors for semantic / hybrid search.
The use of commit-hooks is also very clever (mentioned here in the replies)
I mean these are two completely different approaches for completely different purposes. So saying that git replaces vector database is like saying that that a filing cabinet replaces a search engines.
I was working on memory consolidation and memory pruning 3 years ago (with GPT3 Davinci). Glad to see more people are working towards this.
Really cool work! This is something I’ve been interested in for a while, and I’m excited to try it out. Cheering you on!
Maybe you can use Mercurial for this? You don’t need the complexity of Git for memory.
This is cool! Do you have any plans on releasing it as an MCP server? Would love to add this to my mcp gateway service https://mcp-boss.com/ (shameless plug)
Would have to be productionized, the current repo is a PoC, but if there is enough interest we could def commit some resources to hardening this and adding things like MCP.