Ironic language given that most "engineers" today can't write code without dependency on built in garbage collectors.
Do you really think developers these days are just as good at optimizing for limited system resources as they were in 1986?
"We still run into..."
Who? Maybe you - but most developers aren't running into map/reduce problems every day. And when they do, they certainly aren't coding a solution from scratch.
I think what really happens is that programmer/software engineer profession have become a lot more accessible to the mass. Back in billg days when he started programming, I would bet less than 1000 kids around the country had access to a computer. In 1986, the total number of programmers would've surely been less than 10,000 and most of them worked for MS, IBM, DEC, etc. In the current time, half of SF are software engineers and entrepreneurs :) I guess my point is there are still a lot of people who work on the hard core stuff like Linux/Windows kernels where every byte counts, or JVM/LLVM where data structures/algorithms are rewritten over and over to get a bit more performance. I would say the number of programmers who have to care about the optimizing system resources may be even more than back then, it's just that there are a lot more application developers these days.
I guess my original post was focused on map/reduce problems; I meant it to be more general. We take our site response time very seriously, and therefore our application code and data layer has to be very fast.
Yes boxes are faster and easier to spin up. The 'downside' to that is that many website on the internet are blazingly fast: Google, Amazon, Dropbox, etc. Because they are so fast (or perceptibly fast via tricks), the rest of the web is held to that higher standard. Twenty years ago, things taking a long time on a computer screen with no updates was acceptable. Today it really isn't because there are N other websites that are selling your exact product or experience.
Maybe these days other metrics are more important to optimize for? Ultimately it is about transforming money (resources) into output (more resources) and depending on times different things bottle-neck you there.
Do you really think developers these days are just as good at optimizing for limited system resources as they were in 1986?
"We still run into..." Who? Maybe you - but most developers aren't running into map/reduce problems every day. And when they do, they certainly aren't coding a solution from scratch.