Is this something that can happen? We just ran into this limitation and I really want to keep using pgvectorscale... am exploring other solutions on EKS but RDS would be so much easier. From my reading it seems like this isn't something we can get done as a single AWS customer though.
Thanks. We are already using timescale postgres image for pgvectorscale with some customizations on tsvector and GIN indexes. Would be nice to have bm25 as well. Any specific reason why this was not made open source from the get go or is it the usual phased approach by Timescale (now Tigerdata)? If not, it is a worrying signal as the same could happen with pgvectorscale development as well.
Anyways really appreciate the free offerings by timescale. Really makes things easy.
I am a big fan of cloudflare blogs. The tech blogs are usually highly detailed and there is so much to learn from those.
But this one, interesting but was not a practical choice at all from what I gather reading the blog. The reason given for not using Clickhouse which they are already using for analytics was vague and ambiguous. Clickhouse does support JSON which can be re-written into a more structured table using MV. Aggregation and other performance tuning steps are bread and butter of using Clickhouse.
The decision to go with postgres and learn the already known limitations the hard way and then continue using it by bringing up a new technology (Timescale) does not sound good, assuming that Cloudflare at this point might already have lots of internal tools for monitoring clickhouse clusters.
That's interesting. Personally I did not find it vague and ambiguous.
ClickHouse was fast but required a lot of extra pieces for it to work:
Writing data to Clickhouse
Your service must generate logs in a clear format, using Cap'n Proto or Protocol Buffers. Logs should be written to a socket for logfwdr to transport to PDX, then to a Kafka topic. Use a Concept:Inserter to read from Kafka, batching data to achieve a write rate of less than one batch per second.
Oh. That’s a lot. Including ClickHouse and the WARP client, we’re looking at five boxes to be added to the system diagram.
So it became clear that ClickHouse is a sports car and to get value out of it we had to bring it to a race track, shift into high gear, and drive it at top speed. But we didn’t need a race car — we needed a daily driver for short trips to a grocery store. For our initial launch, we didn’t need millions of inserts per second. We needed something easy to set up, reliable, familiar, and good enough to get us to market. A colleague suggested we just use PostgreSQL, quoting “it can be cranked up” to handle the load we were expecting. So, we took the leap!
PostgreSQL with TimescaleDB did the job. Why overcomplicate things?
> No LangChain, no LangGraph, no LlamaIndex, no CrewAI
Bless you. Using these over complicated abstractions (except CrewAI which I haven't yet checked out) never made sense to me. I understand that LLM is no magic wand and there is a need to make it systematic rather than slapping prompts everywhere. But these frameworks are not the solution to it. Next I will be looking at is Microsofts semantic-kernel. Anybody has any good words for it ?
Built a couple of things with Semantic Kernel. Both some private test projects, but also two customer facing applications and one internal.
It's heavily tilted towards OpenAI and it's offerings (either through OpenAI API or through Azure). However, it works decent enough for other alternatives as well, like: huggingface or ollama. Compared to the others (CrewAI etc). I kind of feel like Semantic Kernel hasn't really solved observe ability yet. Sure you can connect what ever logging/metric solution .Net supports, but it's not as seamless like the others. Semantic Kernel is available in .Net, Java and Python. But it's quite obvious .Net is a lot more polished then the others. Python usually gets new features faster, or at least pocs or previews.
Some learnings from it all:
- It's quite easy to get started with
- I like the distinction between native plugins and textbased ones (if a plugin should run code or not)
- There is a feeling of black magic in the background, in the sense of observe ability
- A bit more manual work to get things in order, compared to the alternatives
- Rapid development, it's quite clear the development team from Microsoft is doing a lot of work with this library
All and all, if you feel comfortable with writing C#, then Semantic Kernel is totally a viable option. If you prefer python over anything else, then I would say llamaindex or langchain is probably a better option (for now).
Thanks. I would have preferred to use Go instead of Python, but somehow the language is not picking up a lot in terms of new LLM frameworks.
As of now, I am using very light weight abstractions over prompts in python and that gets the job done. But, it is way too early and I can see how pipelining multiple LLM calls would need a good library that is not too complex and involved. In the end it is just a API call and you hope for the best result :)
Since you prefer go, you might be interested in one of my pet projects. Where I've glued together some libraries, which lets you basically code all interactions with LLM's through lua. The project is written in go.
Currently it only supports ollama, but I've been thinking about adding support for more providers
As you can see, it's in a very early stage. I'm not a go developer, and I use this repository as a way to explore things both within Ollama and with go.
I'll probably add more things as the time goes by, but it isn't something I hack on every day or for that matter week. Just something to poke around and explore things with.
I'm currently building some stuff with SK using csharp.
Biggest gripes are that APIs have been changing quite a bit over time and the documentation isn't exhaustive in describing what is available.
If you're comfortable digging around in the source, looking at their tests and piecing things together it's pretty solid to build off of and is working well for me thus far.
I really like that there is abstraction enough for me to use other LLMs or implement them if need be.
I echo your apprehension around these abstractions and have built closer towards some of OP's patterns.
I'm working on a very abstraction light package for LLMs called magentic. The goal is to handle the main boilerplate of building with LLMs, but leave the abstractions up to the user. I'd be interested to hear your opinion on it vs others.
Interesting! How does it compare to something like Mirascope? I feel like I'm constantly on the hunt for the slimmest llm interface layer, and mirascope is my most recent attempt.
I wonder why not clickhouse. Cloudflare has already deployed a lot of analytics services based on CH. Would have been nice if they could provide more details on that.
I have been a big fan of C++ and have had used it for about 8-9 years professionally as well as for side projects. The baggage that comes with it is just not worth it. My recommendation for any new project is to go either with Go or Rust as a C++ replacement unless the environment or hardware or resource constraints does not allow you otherwise.
As an individual developer, learning and being an expert in C++ intricacies is not worth the time and effort. Instead learn more about data layout in memory, hardware, language tooling etc and write code accordingly in a much better language as per the _need_ and C++ would really only qualify for very niche requirements.
Really disappointing post from QuestDB. I would have expected them to do some research on how to design CH table schema before doing such kind of benchmark.
The queries used does not take into account the primary key/order by fields. Based on query to be optimized, once could use Projections or MV. Perhaps a bit more work is needed, but that's the clickhouse way of doing it.
What is the difference w.r.t the comparison done by Altinity of clickhouse with timescale ? Clickhouse performed better there for the same test. What gives ?
Thank you. My only nit is the way the ratio (CH/TS) is shown. What is the purpose of that ? It will show a bigger percentage for cases in which TS is better, but lower percentage for cases where CH is giving better results. From the data representation perspective, I do not thinnk that is fair.
The two big things, which we discuss at length in the post, are:
- Altinity (and others) did not enable compression in TimescaleDB (which converts data into columnar storage) and provides improvement in querying historical data because it can retrieve individual columns in compressed format similar to CH
- They didn't explore different batch sizes to help understand how each database is impacted at various batch sizes.
Have you from your side followed all Clickhouse best practices?
Clickhouse design in particular suggests doing ingest request approximately once per second and if you do much more than that when you use it outside of intended usage and if you need that you usually have some sort of queue between whatever produces the data and Clickhouse.
Note the ingest in small batches also can significantly affect query performance
Yep - it's all detailed in the post! The question is how it compares to TimescaleDB, which is an OLTP time-series database that has a lot of other possible use cases (and extensibility). I think it's very fair to explore how smaller batches work since others haven't ever actually shown that (as far as we can see) so that users that would normally be coming from a database like PostgreSQL can understand the impact something like small batches would have.
As for ingest queueing, TSBS does not queue results. We agree, and tell most users that they should queue and batch insert in larger numbers. Not every app is designed that way and so we wanted to understand what that would look like.
But CH did amazingly well regardless of that with batches above 1k-2k and lived up to it's name as a really fast database for ingest!
