yes, with n_val=3 (default) 3 nodes is the minimum. I was merely testing (actually had 3 riak instances on a single box as a "cluster") but with r,w=1 and n_val=1 it was about the same (slow), which was disconcerting. cpu was very high in one instance (others were nearly idle, and IO was low. I did not try dw=0 though (if that would even work). Might be worth a try.
I am going to try bitcask too, to see if eleveldb was the culprit and could be tuned.
Your test isn't valid if it was all on one box - the disk I/O is going to be really nasty with that. Your test would be better on three different machines.
The disk io was pretty low. It is serial get/update/delete (as I mentioned earlier). I even tried moving the data dirs to ramdisk (was plenty of ram), but it didn't help much at all. i/o was certainly not the problem.
It very well could be a configuration issue. I am still trying a few things to see if I can get an improvement.
Interesting. I can vouch for the speed of leveldb on a five node cluster - I'm using it in our startup on 5 dedicated machines with around 12GB of RAM, running on FreeBSD. Riak version is 1.1.4; Riak 1.2 is supposed to have a lot of leveldb refinements.
I am going to try bitcask too, to see if eleveldb was the culprit and could be tuned.