Great article Lukas! A key takeaway here is that Llama was the best LLM. I wouldn't have expected that.

(Disclaimer, Lukas and I are co-founders)

Thanks for your input, I've only tried Chroma a little bit so far and had a pretty good experience. What they also have going for them is a big community on discord that can be helpful.

Yeah, maybe they will.. But for now, the best options are the purpose-built vector databases, so why not use them?

Many of them are open source and you can host them yourself. That would make it more cost effective. Also someone mentioned https://turbopuffer.com/. That seems like a good alternative if you're looking for something economical.

I quickly took a look at the redisearch ANN Benchmarks and they seem to stack up against the others (more or less same level as Milvus) in the comparison when it comes to QPS and Latency.

Yeah, that's the difference we've seen according to the QPS for the ANN Benchmarks. The same story seems to be true for other datasets too. We're looking at a 0.9 recall.

The way you explain hybrid search aligns with my understanding. Pinecone has a good article about it here https://www.pinecone.io/learn/hybrid-search-intro/. From my understanding, all vector DBs support this.

Someone else also pointed out that Vespa was missing. I'll have to look in to it and add it to the article!

Me too! Couldn't find a lot of information on it yet, but I might have to try it myself to get some benchmarks

Turbopuffer looks like something I would consider. And the pricing looks to be lowest on the list from what I can see

Emil if you email me at info@turbopuffer.com I can let you into the alpha :)