By the looks on the code this isn't really a graph database. It's more a graph database emulation built on top of a SQL database. For distribution it uses sharding so it isn't really distributed either, at least not like Cassandra or other distributed databases.
I think Redis would perform a lot better than SQL for graph like structures - since sets are a native datatype in Redis. And you can go A LONG way with just one Redis database (currently we are storing over 20 million keys in our Redis database and I know some that are storing 100 million keys on _one_ server). And with the new Redis VM coming up, I would guess that scalability of Redis is going to be even better.
Other than this, neo4j seems very interesting and would probably also have been a better choice than using a relational database.
You can go a long way with a SQL database too. I'm writing up an article about getting the two billion words of Wikipedia into an inverted index in a SQLite database entirely in memory, and that's another other order of magnitude bigger.
If you only know how to use a hammer then everything else looks like a nail. In other words, re-implementing an inverted index in SQL is waste of time when you can use tools like Sphinx and Lucene - which are highly optimized to do inverted indexes and that can easily handle 2 billion words. The same can be said about FlockDB - it's possible to emulate a graph database, but is the effort really worth it when there are such tools like Redis and neo4j which seem to be optimized for graph like structures.
Hmmm... This looks to only store first order relations efficiently, its seems that to traverse many nodes, you would need to repeatedly query the database (e.g. I can only get my friends, not the friends of my friends etc...). This severely limits the use for most problem domains you would want to use a graph Db for. Still, possibly useful if you have to solve a problem that looks alot like twitter's.
Once you start getting out to second order it becomes a much more complicated (and interesting) problem -- one that I've been kicking around for a while.
Data-locality is the kicker in a distributed graph database; when doing traversals that cross multiple nodes you need to have a partitioning scheme that coordinates with your traversal algorithms so that you need the minimum number of machine-to-machine hops in a multi-level traversal. Getting that right is far more difficult than traditional database sharding.
Yeah sure.... it is much harder and involves minimizing the number of relations across shard boundaries. Not easy. However, to call a system that only allows depth 1 traversals a 'graph database' is slightly pushing the definition. To me, its more a "key value database with relations between keys".
Everyone has different requirements though, if depth 1 and huge scale are what you need, flock db might be for you.
Note that it's in the process of being released: it's as yet unusable by outsiders. Honestly, I did not expect this to make Hacker News so soon. :(
I have not used Neo4J first hand. It has really cool features, but it is not a distributed database and has expensive memory usage. FlockDB is distributed, uses little memory, and has a very limited feature-set that is highly highly optimized for OLTP. It's not really an apples/apples comparison. Theoretically, Neo4J could be used as a back-end data-store in FlockDB.
I agree that it's not apples/apples. From the first few minutes, I think the main strength of Neo4j is the rich ecosystem and functionality on top of it, and the fact that it stores an infinite-levels deep graph. In comparison FlockDB stores one level (e.g. user -> followers). The main strength of FlockDB seems to be that it has built-in distribution, which is something we're working on for Neo4j but it's not yet generally available.
All this of course based on just a quick glance, so I may come back all the wiser and revise my opinion later. :)
Well, the data model seems very similar to Redis' from first glances [1], but FlockDB certainly seems to have completely different durability characteristics. So even if they started anew today they may end up building their own.
1] Which would make FlockDB less a graph db and more a key-value store with social network semantics for the values.
"Lines of doesn't say anything" is one of those flawed mantras that people keep repeating as an overreaction to the too often used assumption that it's the most important metric.
I think Redis would perform a lot better than SQL for graph like structures - since sets are a native datatype in Redis. And you can go A LONG way with just one Redis database (currently we are storing over 20 million keys in our Redis database and I know some that are storing 100 million keys on _one_ server). And with the new Redis VM coming up, I would guess that scalability of Redis is going to be even better.
Other than this, neo4j seems very interesting and would probably also have been a better choice than using a relational database.