Good point. I will put a MIT License file in there shortly.

I wanted a cross platform way to test slow connections with a single command line parameter. Whether it be Linux, FreeBSD, or OSX (maybe Windows (haven't tested :-P)), it is easy to setup.

iptables or putting nginx with rate limits in front of Fake S3 would be a more powerful approach, but also harder to get going.

Excellent points.

We work on the idea of different stages in the test and development pipeline. At different stages mock objects make sense, and at other stages having something like Fake S3 makes more sense.

For testing, the first stage would be unit testing. At that stage it is best to mock out your S3 interactions (with something like VCR or WebMock) and use an OO approach to wrap your persistence, so you could swap out S3 with another persistence engine without breaking APIs.

The second stage for us is integration testing where you might have multiple machines testing across the network. In this situation, I think it is great to have real network requests happening rather than mock requests. Also you can deal with real files (especially important with media files like images and video).

The last stage is taking out Fake S3 and using a true S3 connection to ensure that everything does work on a production environment (cuz Fake S3 could be faking you out, especially on things like authentication and versioning). We do that by launching a stage cluster and running a set of integration tests on that before doing a production release. Ideally, the first and second stages catch any errors before you start doing tests against the real AWS services.

As for the development pipeline, being able to work with real assets while you are making mobile or web interfaces is really useful, as well as simulating latency to see how interfaces respond when under a slow network connection is something that would be difficult to truly mock.

Awesome, thanks for the extra info. I think your setup sounds really good :)

it's fine to mock, but for serious s3 users you need to emulate the exact behavior or you are setting up your users for failure.

Great idea. I like the idea of a command line flag (like the rate limit flag) to run it with a percentage failure rate or something along those lines.

Swift is very powerful piece of technology, but it is also more involved to setup. Curious to try RiakCS as well and see how it compares to Swift for running production level S3 object storage.

http://devstack.org has a script to deploy OpenStack in two lines (git clone the repo, then run the script)

Looks like I need to post my blog post about how to set up Swift really easily!

My twitter handle is @jubos. I would love to see how you approach it.

Isn't RiakCS an online service, like S3 itself? That's what I understand from the basho page...

Thanks! I haven't but let me know if works.

Spool is still here, and still transcoding and caching videos. We got some new stuff coming down the pipe, so stay tuned.

I just started using Backblaze last week, and so far I have been very pleased. Even with Time Machine, I found I rarely hooked up my external hard drive at home, so this gives me a little more peace of mind, not to mention continuous backup during the day at work, in case my laptop gets jacked on the commute home.

Just curious, how does the Lua runtime perform with bitmap operations?

hi, one of the makers of Spool here. Thanks for kicking the tires on it. Now that our coverage is up, we are optimizing for speed now, so those wait times will come down.

Hi Jubos,

I dont know how to contact you/message you on HN but there's one thing that has gotten a little annoying on spool; when you get to the bottom of the article - it brings up a big spool banner and this gets in the way of finishing up reading the article. I'm not sure if this is a bug or a feature but it's a little annoying :-/