An honest confession from someone inspired by these comparisons. I've been following these benchmarks very closely right from round one and ever since I've been waiting to see my favorite framework, Ruby on Rails perform decently to some extent. I waited till the last round to see some improvements and while I DID see some marginal improvements, it wasn't as expected.
This then provoked me to do some basic math. Imagine a real world startup scenario wherein you need to your application to deliver API responses (JSON) or even ordinary HTTP responses. If you use a framework from the bottom of the list (like Rails), then you may be able to serve, say, 'X' customers on your site using your application at a time. Now, if you use something from the top of the list, you may be able to serve roughly about 20 times (20X) more number of customers on a single server. This not only means less cost, it also means you can worry about scaling and stuff much much later than you would have to, unlike on a low performant framework.
For this reason, I decided to conduct an experiment for myself. Pick up a framework from the top of the benchmarks and experiment with the learning curve and the language. Well, after carefully evaluating, I decided JAVA wasn't for me (too much of a beast). But Scala seemed to hit the sweet spot with excellent language (1 line of Scala = 10 lines of Java on an average) features along with the trust of the JVM.[1]
So, I chose Scala and started experimenting with it. I started with the play framework and was not fond of certain decisions that were made within the framework. Also, the play framework is nothing like Ruby on Rails (but it claims to be so) and was more of self-serve and also quite heavy for my taste. Hence, I went even farther up the list and chose Scalatra. Generally, framework selection is a matter of personal preference and each one should choose whatever aligns with their philosophy. For me, it was scalatra.
Frankly, I haven't been able to ship out features at the speed at which I used to, with Rails, but there was something very important that I learnt too - Don't get comfortable with full stack frameworks without understanding what goes underneath. Rails suddenly seemed like an enemy to me because it was doing everything so magically that I literally had NO IDEA until I had to implement them myself with Scalatra. For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch???? I knew none of these, until I had to implement them myself for my application on top of Scalatra. It was seriously a pain because for the first one week, I did absolutely nothing, in terms of progress with features. But later, I started loving this way of development rather than relying on 'magic' frameworks. Now 'magic' seems scary for me because I cannot actually see what's happening underneath.
So, to cut a long story short, Should you choose the fastest framework? Or should you follow the 'ship it first' policy with an average framework??
My advice - Start with a full stack framework, something like Rails or Django, but also in parallel, try to understand how every bit of your framework works and try to implement it into a micro framework based on something from the top of the benchmarks (like Go, Scala, etc.). Most importantly LEARN. Something new everyday! And slowly shift your development towards these high performant frameworks as and when you see fit.
> For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch????
If I answered yes to literally all of these questions, am I still allowed to use rails?
Allowed and encouraged! It's good to both have standard tools that not everybody needs to reimplement differently (and often poorly) and to understand how those tools work. Everyone should take the time to get at least highly acquainted with the libraries they depend on. I suppose this was a good way for your parent to learn that lesson, but it's also a good example of why benchmarks aren't everything for frameworks; their design and community matter a lot.
I had similar ambitions, and I wanted to play with the top performers, so I started looking at the Java (gemini) ecosystem.
And after an hour of surfing I remembered why I didn't follow through the last time I had this idea. The Java ecosystem is complex, I mean really complex.
Here are some of the terms I came by that look like I should grasp before trying to make something: servlet (various versions), servlet container (resin, tomcat, jetty, ...) , OSGi, jar, war, wab, jboss, glassfish, java standard/enterprise edition, where are classes being imported from, configuration files for different containers, JNDI, classpath...
I'm sorry you spent time trying to get started with Gemini, because it's our internal framework that isn't released yet. That would have been frustrating for you!
From our "Motivations" section: http://www.techempower.com/benchmarks/#section=motivation "Why include this Gemini framework I've never heard of?" We have included our in-house Java web framework, Gemini, in our tests. We've done so because it's of interest to us. You can consider it a stand-in for any relatively lightweight minimal-locking Java framework. While we're proud of how it performs among the well-established field, this exercise is not about Gemini. We routinely use other frameworks on client projects and we want this data to inform our recommendations for new projects.
Have you considered starting with Grails? It's a much easier jump into the "java-ish" world. The main benefit is you can choose to run a grails application and rely on java jars for when you need performance boosts.
We've been using Groovy internally for a few years for prototyping and experimentation. I thought about slowly introducing Grails as an alternative to Spring for some projects. My experience was that it vaguely reminded me of Rails but needed a lot more polish. I had a huge issue trying to chase down archaic stacktraces and found that there is little developer momentum pushing it forward. I usually found myself googling and reading random blogs to figure out simple things I couldn't infer from the docs. Where as Rails magically makes things work, I found that a lot of times Grails made things magically not work. I wouldn't personally suggest it for anything customer facing.
You could use JRuby and Rails if you need the JVM.
Groovy's OK for wraparounds testing and/or running Java classes, what it was originally intended for, bringing closures and terser list/map syntax (altho JRuby et al have those too). It's all rather slooooow though. Grails uses Groovy's MOP which was added later, but I don't use them much. I toyed with Groovy's interceptors and categories when they were added, even wrote notes on them, but they broke in the upgrade from 1.5 to 1.6. There's a lot of such cruft in Groovy which just lapses from version to version through lack of use and support. They brought in AST transforms to get programmers to extend Groovy's functionality, but when they do, the Groovy managers wil swipe the code and pass it off as their own in the next version of Groovy (e.g. Groovy++ written by Alex Tkachman was cloned as the static compilation in Groovy 2.0 and the static traits in the upcoming Groovy 2.2).
tldr... Use grOOvy for quickies handling Java classes, and for Grails where it's required, but look at a serious programming language for anything larger.
In previous rounds there were some discussions that you were considering open sourcing Gemini. Is that still on the table or have you decided against it?
gog, I totally agree that the documentation for the Java ecosystem is awful. My theory is that the relevant information for today is lost under 18 years of noise.
What I'd like to see: Java should be renamed with every major version [1], in much the same way Ubuntu, OS X, and Android are named.
It doesn't appear that there's been any real speedup on the Ruby on Rails results for the last several rounds, so... why is that? Does nobody in the community care about speed? Is there no way to speed up Rails? Is it not clear where the bottlenecks are? Are the profiling tools not good enough? I'd sincerely like to know if someone has taken on the task and just been frustrated, or if no one has actually tried.
We have received numerous pull requests from a spectrum of communities, but the Ruby community has been fairly quiet on that front. For example, not a whole lot has changed with the Rails test since March aside from us adding implementations of the newer test types:
I believe the '1 JVM = 10 Thins' remark you link to refers to the amount of servers they needed to run. Not the amount of code. Also Thin is a Ruby webserver, not Scala nor Java.
I've been on the other side of that equation. Building all that stuff by hand leads to incredible waste. Bike shedding, reinventing the wheel and substandard results.
Do I like Active Record? No, not really. Do I want to fight with the entire development team to force what works best for me onto everyone else? No, not really. If we are productive with AR, I'll accept it.
This then provoked me to do some basic math. Imagine a real world startup scenario wherein you need to your application to deliver API responses (JSON) or even ordinary HTTP responses. If you use a framework from the bottom of the list (like Rails), then you may be able to serve, say, 'X' customers on your site using your application at a time. Now, if you use something from the top of the list, you may be able to serve roughly about 20 times (20X) more number of customers on a single server. This not only means less cost, it also means you can worry about scaling and stuff much much later than you would have to, unlike on a low performant framework.
For this reason, I decided to conduct an experiment for myself. Pick up a framework from the top of the benchmarks and experiment with the learning curve and the language. Well, after carefully evaluating, I decided JAVA wasn't for me (too much of a beast). But Scala seemed to hit the sweet spot with excellent language (1 line of Scala = 10 lines of Java on an average) features along with the trust of the JVM.[1]
So, I chose Scala and started experimenting with it. I started with the play framework and was not fond of certain decisions that were made within the framework. Also, the play framework is nothing like Ruby on Rails (but it claims to be so) and was more of self-serve and also quite heavy for my taste. Hence, I went even farther up the list and chose Scalatra. Generally, framework selection is a matter of personal preference and each one should choose whatever aligns with their philosophy. For me, it was scalatra.
Frankly, I haven't been able to ship out features at the speed at which I used to, with Rails, but there was something very important that I learnt too - Don't get comfortable with full stack frameworks without understanding what goes underneath. Rails suddenly seemed like an enemy to me because it was doing everything so magically that I literally had NO IDEA until I had to implement them myself with Scalatra. For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch???? I knew none of these, until I had to implement them myself for my application on top of Scalatra. It was seriously a pain because for the first one week, I did absolutely nothing, in terms of progress with features. But later, I started loving this way of development rather than relying on 'magic' frameworks. Now 'magic' seems scary for me because I cannot actually see what's happening underneath.
So, to cut a long story short, Should you choose the fastest framework? Or should you follow the 'ship it first' policy with an average framework??
My advice - Start with a full stack framework, something like Rails or Django, but also in parallel, try to understand how every bit of your framework works and try to implement it into a micro framework based on something from the top of the benchmarks (like Go, Scala, etc.). Most importantly LEARN. Something new everyday! And slowly shift your development towards these high performant frameworks as and when you see fit.
This is just based on my experience :)
[1] 1 JVM = 10 Thins, Source: http://www.slideshare.net/pcalcado/from-a-monolithic-ruby-on...