Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I had actually tried to do this exact thing. Problem A is that you can't execute very much in a clients browser at any one time. Okay, so you make the jobs smaller and fetch more. Problem B is that it turns out that by the time you've pulled the data from disk, shipped it to the browser and back, put it back to disk, and done the same cycle for the reduce, it ends up being cheaper to stream 64M HDFS blocks around EC2.

That doesn't even get in to verifying results from an untrusted client.

Here's my half-implemented proof of concept from a while back that runs on AppEngine: http://github.com/markchadwick/emarer/tree/master

Slightly different implementation, but the same idea (which I think is a very cool idea!).



You can extend the amount of processing time available on the client by storing intermediate results in window.name (which gives you up to 2MB of semi persistent storage)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: