I totally agree that Ruby + R makes a great data toolbox. I use Ruby + R for almost all of my day to day exploratory data analysis work.
As far as python goes I think python has become more popular simply because it has more community around data applications. Unfortunately a lot of people view ruby as just Rails. I think academia's adoption of python has also helped it grow into a data analysis language.
Really any MR you do with python you could do with Ruby as it all uses hadoop streaming.
I did look at GSL :). I decided to write it by hand because I wanted to help people understand the underlying math. I find that too many people use libraries without understand the math which can become problematic especially when performing statistical analysis.
Thanks for pointing out GSL. I appreciate the feedback.
It's definitely good to understand the underlying math, and it's a good article on "this is how to translate a formula into Ruby code", but given that the article is positioned as "We needed to solve this problem, and this is how we solved it", it seems like it'd make more sense to focus on solving it with the least work and the best performance, which is why I mentioned GSL.
It's great to see other people doing statistical work in Ruby, though, so please don't let my criticism keep you from continuing to do it! :)
I appreciate the feedback, especially when it is constructive criticism. It helps me understand how other people were interpreting the post. Perhaps I got the positioning slightly wrong as I wanted it to be more about teaching the basic math and how to translate that into Ruby. I will try and get the positioning better next time.
While we do some stats in Ruby it definitely doesn't represent the entire "this is how we solved it". In fact we use a large amount of R and Java to solve all our statistics problems.
That is exactly what I was going for. Basic algorithm yourself to help people understand the math but you wouldn't want to use this code in a production system.
In fact I perform the vast majority of my statistical analysis in R.
Ruby is just a fun language to implement basic statistical algorithms in and it the Ruby community as a whole doesn't hasn't put a lot of emphasis on stats.
There is a growing community with quite a bit of support surrounding scala particularly for web development. Lift is well documented and in production at Foursquare which is vote of confidence. The Guardian is also switching to scala as described here http://www.infoq.com/articles/guardian_scala. Plus Twitter, Yammer and a bunch of others are also using scala for "web related" development.
Overall I think as it exists now Scala has the best community particularly around web development. Much like the Django v Rails debate try them out and see what you like.
I know personally scala is second on my list behind Ruby.
2011 is going to be the another year all about web 2.0. I think there is going to be an increase in popularity for mobile web development using html5, css3, javascript and frameworks such as PhoneGap and Sencha Touch. The ever increasing number of mobile platforms will have developers looking for ways to maximize their exposure with minimal effort.
I also think there is going to another generation of social networking apps that build on top of Facebook and Twitter giving us new ways to analyze and use social data.
Overall I would say get yourself into some web development
Just by doing some coding. I have experience developing ruby on rails and iPhone applications. I was just looking for a way to get involved in the startup scene in SD
I think there are many things that you need to consider when deciding on a technology. One is preexisting skill, if you are already a .Net C# person then go with what you know. That being said rails is free and can be deployed on *nix which is also free.
Dont get caught up too much in worrying what other people are using. You initial technology stack will not make or break your company as you can always iterate to another technology stack at a later date if you wish.
As far as python goes I think python has become more popular simply because it has more community around data applications. Unfortunately a lot of people view ruby as just Rails. I think academia's adoption of python has also helped it grow into a data analysis language.
Really any MR you do with python you could do with Ruby as it all uses hadoop streaming.