You're not wrong about VC-funded database arc, but what history are you even talking about?
I am sure you understand that I have absolutely nothing to do with Scylla's licensing. I have not worked there for four years nor was I ever in a position there that I would even had that opportunity to influence such decisions.
I am also sure you understand that Scylla's development model was completely different: they had AGPL license and contributors had to sign a CLA, which is why they were able to relicense in the first place. Turso is MIT licensed and there's no barrier to contributing and, therefore, already a much bigger contributor base.
I fully understand the scepticism, but you're mistaken about the open source history of Turso's founders.
The project is MIT licensed with a growing community of contributors. It does not even matter how long the company lives, all that matters is that some of the core contributors live.
The actual reality is that I personally started the project because its synchronous architecture is holding back performance. You can read all about it in https://penberg.org/papers/penberg-edgesys24.pdf. The design is literally the next evolution of SQLite's architecture.
Remember, that argument to `findOne()` is an arrow function. You need a compiler to turn it into a efficient, deferred query; otherwise you need to _evaluate_ the function at runtime.
This is the disconnect I've found in my self-taught compiler education journey. Every time I try to tackle this topic, I get bogged down by parsers and lexers. I already have that structure.
Users.find(x => x.name == "Bob").
My programming language has done that for me. What's the next phase? What data structures do I need to maintain to construct these DSLs? How do I optimize them?
I can fight my way through this but I wish I just had the theory and making this kind of stuff easier. It just comes up so often in the problems that I solve and I'd like to reach for this tool more often.
Perhaps you are looking for macros? Rust had a substantial macro system while being low-level enough to fit into most architectures. I haven’t implemented a macro but I’ve used many. Have you looked there?
As I was writing my comment, I think maybe I should start with DSL, but this isn't conclusive. Spark, Scalding, LINQ, RxJava, even Slick are examples of the types of DSLs I'd like to understand how to construct. They're essentially compilers.
Looking at the code for these technologies it's incredible how complex it is to implement the nice DSL that they expose. How much of that is accidental complexity?
I don't know if SQLite + Raft is going to be the next Postgres, but it's certainly an interesting solution for a variety of use cases where you don't need your data _partitioned_ on multiple machines (but still want it to be replicated for availability and fault tolerance).
So thread-per-core is not just about eliminating context switches. It's also about partitioning application-level data to reduce inter-core synchronization to let speculative, out-of-order cores run independently as much as possible. Don't get me wrong, user-level threads are great, and arguably much simpler programming model than async/await. But they're not _the solution_ either.
User-level threads do not solve the problem of synchronization and data movement. That is, with a “thread-per-core” model, you eliminate most needs to synchronize between multiple CPU cores, and, therefore, eliminate the overhead of acquiring and releasing a lock and allow the out-of-order CPU cores run at full speed. Furthermore, “thread-per-core” model ensures that memory accesses are always CPU local, which eliminates the need to move data between CPU caches, which eliminates the expensive CPU cache consistency protocol (that makes scaling to multiple cores more difficult).
That said, I am not claiming thread-per-core is _the solution_ either, just saying that if you can partition data at application-level, you can make things run plenty fast. Of course, you’re also exposing yourself to other issues, like “hot shards” where some CPUs get disproportionately more work than others. However, as we scale to more and more cores, it seems inevitable that we must partition our systems at some level.
It's pretty much common knowledge that if you want linear performance scaling on a multiprocessor then each workload has to be effectively independent from the others. Synchronization and communication is expensive and only takes away from your per core performance but it never makes a given core faster. So yes, sharding is a critical part of this "thread per core" architecture.
To be fair, concurrent writes do not scale, but single writers multiple readers work just fine, so you only need to an hard partition of writers and can allow all threads (or the appropriate NUMA subset) to access any data.
binding user level threads to hardware cores without preemption is completely equivalent to the thread-per-core + coroutines, except that user level threads allow deep stacks (with all the benefits and issues that it implies).
In fact a well designed generic executor can be completely oblivious to whether it is scheduling plain closures, async functions or fibers. See for example boost.asio.
Excellent question! I think VoltDB, for example, uses the same application-level data partitioning approach, but with processes instead of threads. One advantage of the "thread-per-core" approach is that it allows fast communication between shards because the underlying threads share the same address space. In contrast, processes need to use a more heavy-weight approach of inter-process communication (IPC). Of course, process-based systems will have a reliability advantage, because processes cannot easily mess with each others state.
AGPL treats "networked access" as "distribution", not "linking".
With the GPL, you are required to make your modifications available as GPL if you distribute the modified program. This creates a loophole for ASPs, which can modify GPL’d code, but because they offer it as a service (and don’t distribute the program), they don’t have to share the modifications. AGPL closes this loophole by explicitly saying that network access is considered as “distribution”.
So no, you don't need to make your web application AGPL. Just like you don't make all your applications running on Linux GPL just because they happen to communicate with the kernel over system calls. Copyleft licenses are relevant only if your code is a derivative work of the original.
The licenses you are talking about, such as MIT or Apache, are called _permissive licenses_ whereas AGPL is a _copyleft license_. However, both types of licenses can be open source licenses, if they fill the OSI requirements.
I am sure you understand that I have absolutely nothing to do with Scylla's licensing. I have not worked there for four years nor was I ever in a position there that I would even had that opportunity to influence such decisions.
I am also sure you understand that Scylla's development model was completely different: they had AGPL license and contributors had to sign a CLA, which is why they were able to relicense in the first place. Turso is MIT licensed and there's no barrier to contributing and, therefore, already a much bigger contributor base.
I fully understand the scepticism, but you're mistaken about the open source history of Turso's founders.