While it presents txtai as a very neat framework for calculating and searching for embeddings the article doesn’t seem to mention anything about a database server, except for the drawing at top.
The code looks like it is doing an in-memory search.
When searching an embedding space, it is quite common to look for the closest angle of the query vector against amongst the search vectors.
That is a calculation I don’t know how to make a sql server perform and given the headline I thought the article would show a neat trick for doing this.
At a high level, txtai uses a similar clause embedded in SQL statements. For example, "SELECT id, text, score FROM txtai WHERE similar('feel good story') AND text LIKE '%good%'". This statement is parsed and the similar clause runs against the approximate nearest neighbor index. The result ids are then loaded into a temporary table and the SQL statement is dynamically rewritten to change the similar clause into "id IN <temporary table>".
I’m late to the party; I just learned about embeddings. I’ve been playing with lexvec this week. I feel like I’ve found a book of working magic spells.
In 10 lines of code you can import a transformer and embed your phrase/word. Then you can do prediction or similarity search really easy. It's a superpower, true.
I’m not clear on “where” this sits, is it a framework for turning text into embeddings? Is it the database and serving layer (like Milvus, Vald, Pineconr, etc)? Is it just a Python package for generating and serving vector search in-process?
txtai uses transformers to transform data (text, images, audio) into embeddings. Those embeddings are then loaded into an approximate nearest neighbor index for search. On top of that, content is loaded into a relational database to support SQL based filtering. It's trying to get the best of both vector/similarity search alongside standard structured search using a SQL syntax.
Components can be split up, for example there could be a server that vectorizes text into embeddings and another server that hosts the indexes.
There is also a pipeline and workflow framework (https://neuml.github.io/txtai/workflow/). This component has modules that assists with splitting data, transforming, summarizing, translating, parsing tabular content. Workflows can be used purely for transformations or as a driver to load data.
I've primarily focused on single query times, which have averaged around 5ms - 25ms depending on the size of the index. Queries can be batched so query times wouldn't increase linearly. You could batch quite a few and still get subsecond response times.
All three libraries are approximate nearest neighbors but I know at least for Faiss, it can be configured to effectively be a precise query.
The code looks like it is doing an in-memory search.
When searching an embedding space, it is quite common to look for the closest angle of the query vector against amongst the search vectors.
That is a calculation I don’t know how to make a sql server perform and given the headline I thought the article would show a neat trick for doing this.