Geoffrey Hinton's tech-talk in 2007 (!) at Google is a great watch (with a heavy dose of technical jargon [0] plus some dry British humour interlaced throughout). He explains digit recognition (vs SVM) [1], document classification (vs LSH), and breifly summarises image classification [2] problems and how they were solved: https://youtu.be/AyzOUbkUf3M
You could instantly see the results he presents were way better than what was state of the art at that time. Amazing.
Although 3b1b's videos are awesome, his NN videos are heavily inspired by this: http://neuralnetworksanddeeplearning.com/. Even the code on his github page is taken from there.
No invention occurs in a vacuum, most creations are recombinant effects.The pieces of the puzzle might very well have existed, but Hinton deservedly receives credit for putting the various pieces together for backprop IMHO.
Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.
Wright brothers were most likely not the first to achieve heavier-than-air powered flight. But they made the first controlled, sustained flight. Controlled flight was the real breakthrough in aviation.
It's the same with Hinton et. all. They were among many other pioneers, but what really sets them apart is that they made it all work. They also analyzed why it works.
> but what really sets them apart is that they made it all work [citation needed]
Werbos and others were very aware that the method worked and also of the implications of it:
> This approach makes it possible to develop generalized, adaptive artificial intelligence, capable of achieving results comparable to what is discussed in science fiction
The Hinton cartel happened to be in the right condition that allowed them to ignore the politically correct BS starting at the turn of the century which rendered the study of neurons, IQ and intelligence massively unpopular; and ruthless enough to only cite one another and call themselves the "fathers of deep learning" rather than actually citing the originators of these ideas.
People deserve credit for taking an idea and running with it. Without Hinton's efforts (in 1987, 2006, or 2012), deep learning would have never seen the explosive growth.
"would have never..." it's not possible to make such an assertion unless you have access to at least one copy of this universe that doesn't include him. Maybe in all the other universes the exact same thing happened except someone else did it. Maybe in some of those universes, without him and his popularizing of this technique, some other better technique flourished. Who knows?
Stating "would never have" is nothing more than telling a story.
> People deserve credit for taking an idea and running with it.
Yes, as popularizers.
> deep learning would have never seen the explosive growth
This is extremely unlikely to be true. I think the explosive growth occured because neurons and intelligence were unpopular in the 2000s because of PC, deferring progress in this area to occur explosively later. The Hinton cartel were not the only people who were aware that AI research "has become silly" (in his own words).
PR is what all science is about. Without communicating ideas, science is not useful. The trio above managed to communicate deep learning ideas to the wider vision and NLP community such as by collaborating with them to come up with benchmarks which would prove usefulness of deep learning, but also to industry and general public. Schmidhuber did very little on these fronts, and even his technical contributions while laudable do not compare with the awardee's.
Fascinating! Thank you for sharing. The issues around the assignment of credit is as old as science and the concept of discovery; reading "The Invention of Science: A New History of the Scientific Revolution" by David Wootton really opened my eyes into how this is actually a fundamental feature of our modern conception of scientific discovery (sounds counter-intuitive.. I know!). It's like a societal "bug".
This is well deserved, but IMO it also signals an upcoming AI winter. This cycle has happened several times before:
1) Some scrappy researcher on a low budget develops a new AI technology that shows promise in a specific area.
2) More researchers take that idea and successfully apply it more broadly.
3) Massive investment in research happens, pushing PhD candidates into extracting every possible nuance out of that technology.
4) Researchers, encouraged by the broad success, begin to think we've found the one true key in the quest towards general intelligence. Suddenly all the researchers are Neats. The research funding is now mostly businesses instead of governments and non-profits. They grow overconfident that it can be used for virtually anything, funding everything in sight.
5) Starry-eyed futurists start lionizing the founders of the new revolution as geniuses, publishing flowery bullshit about how the world will be forever changed and how the General AI revolution is only 5-10 years away.
6) Surprise! It actually can't be used that broadly. Improvements stall, and CEOs start to realize that they can't just dump data in and get money out. It becomes a commercial disappointment, triggering disinvestment.
7) The Neats, disappointed that their glamorous and simplistic theory of general intelligence, start to disappear. Newspapers begin to make fun of all of the visionaries, comparing them to the people that said flying cars were 5-10 years away.
8) A handful of Scruffies take over the now low-funding AI winter, working hard on minimal budgets until a new breakthrough is found and we return back to #1.
This article is showing that we're solidly within stage 5, and we're already seeing signs of stage 6. When everybody is buying, it's time to sell.
There's good reason to think that the AI Winter / AI Spring cycle is done. AI techniques are now creating so much real world value (more so than in the 80's / 90's) that there may well not be any more "AI Winter" events. Or perhaps they will be, but they'll be less pronounced. Maybe an "AI Fall" instead of "AI Winter".
Part of that too, is that I think people now realize that narrow AI is sufficient to create tremendous value, and that it doesn't necessarily matter if the "AGI breakthrough" happens anytime soon or not.
It's hard to be sure, but I don't see the kind of collapse that happened in the past happening anytime soon.
AI winter doesn't mean people stop using it. AI winter and spring aren't a phenomenon specific to AI...in essence it's the same dynamic found in economic bubbles. In almost every economic bubble we see actual economic benefit underlying the hype, but the hype grows bigger than the benefit can account for, triggering eventual collapse of the hype. The thing that you should notice is that when the bubble collapses we don't stop buying, we just stop overinvesting.
We haven't stopped buying tulips, trains, stocks, technology, or real estate. And just as well, we haven't stopped using symbolic AI, single-layer neural nets, ensemble models, expert systems, or logic programming languages. The new topological enhancements to Neural Nets won't ever go away either...but that doesn't mean we won't see a drop in investment once the general public realizes that your neural nets aren't going to learn how to do double-entry accounting any time soon. The AI winter isn't characterized by the technology going away, it is characterized by lofty idealism being shattered and investment dropping back to reflect reality.
AI winter doesn't mean people stop using it. ... The thing that you should notice is that when the bubble collapses we don't stop buying, we just stop overinvesting.
Right, but that doesn't match the way I feel the term "AI Winter" has been used. Now I could be mis-interpreting things, but I've always looked at an "AI Winter" as a period of underinvestment, created as an over-reaction to the mechanics you're referring to.
but that doesn't mean we won't see a drop in investment once the general public realizes that your neural nets aren't going to learn how to do double-entry accounting any time soon.
Right, but again, I don't think most people use "AI Winter" to mean a simple "drop in investment". If it were something that straightforward, there would be no need for the "Winter" metaphor.
And that's why I say there may indeed be a drop... an "AI Fall" if you will, that still represents a pull-back of sorts, but perhaps just a less pronounced and extended pullback like we've seen in the past.
Thinking to another domain, we were in a "design winter" in the 90s, until Apple showed with the iPod that there was value there. Then for a while there were many companies creating "real world value" by giving designers authorship.
Now it feels to me we are in a design winter again: there are still a lot of designers on staff, but they are relegated to a "touch up" role... They have been moved ahead of implementation (which is good) but they are expected to "touch up" concepts generated by management in a matter of days. Very similar to when designers would get a few days to clean up a UI that was already implemented.
Sort of off topic, but just and example that widespread use and business value doesn't mean something can't fall back into a winter of sorts.
The assumption here is that 6) "can't be used broadly" is true. What evidence do you have for this? It turns out this technology is already being used everywhere: text translation, image recognition, autonomous vehicles, text generation and so on. It's already generating huge value which is why so many companies are investing so much in it. There isn't going to be any Winter coming anytime soon.
You've identified four areas where it excels. Only a few billion more areas to go. And sure, we can now have algorithms that can categorize dogs based on collections of pixels better than humans can. Get back to me when they've figured out double entry accounting, splitting atoms, building bridges and spaceships, or improve upon TCP-IP.
This is a computer science award, nobody is claiming that AI will revolutionize the fields you mention (which are at best loosely related to CS). It doesn't have to be useful literally everywhere to have value.
This is mildly related, Matt Levine seems more popular around these parts nowadays, he had a bit a few months ago that I find resonates when I see people who seem to constantly seem to speak vaguely in stock market terms and bubbles and soforth. Not really directed to you, more to the parent.
>On the other hand, if you bet that stocks will go down, you have some compensating psychic rewards. For one thing, occasionally stocks will go down, and you will be praised for your prescience in predicting the crash, and the people who were long will be mocked for their complacency. How smart you will feel!
>For another thing, even if stocks haven’t gone down, you get to borrow psychically, as it were, against that future moment of glory. You can just go around sort of saying “this is unsustainable and eventually stocks will go down and I will be praised for my prescience,” and people will be surprisingly willing to say “yes that’s correct, I admire your hypothetical prescience.” Particularly since the 2008 financial crisis, financial markets—and financial media—have a strongly entrenched narrative of prescient bears and complacent bulls, a widespread sense that any rising market is suspect and that the cynical view is always the smart one.
It's just weird to me there's always people looking to figure out a way to call the top on absolutely everything.
Yes, people actually are claiming that these new NNet advancements are the key that will unlock AGI, which by its very nature implies that they will be as advanced, or more advanced, than humans in all forms of intelligence. They're lightyears away from that, but now the general public is buying into the hype and thinking it's a decade or two away.
NNets are already a disappointment to nearly any applied researcher that isn't sitting on petabytes of data. It won't be long before the CEOs realize it and put their research funding somewhere else.
Yes, people actually are claiming that these new NNet advancements are the key that will unlock AGI, which by its very nature implies that they will be as advanced, or more advanced, than humans in all forms of intelligence.
Who are "people" in this context? Are you talking about the general public, or misinformed journalists who are writing about things they don't understand? Because from what I've seen, by and large, the researchers working on Deep Learning are not claiming that DL (alone) is sufficient to achieve AGI, nor do they posit that AGI is anywhere close to reality. Read, for example, Martin Ford's book Architects of Intelligence[1] and note how the various researchers interviewed talk about AGI. That list, BTW, includes all three of LeCun, Hinton, and Bengio, as well as many others.
Let's use a concrete example of technology being overpromised to the public - self-driving cars. Now there is some debate as to whether or not general AI needs to be invented in order to make self-driving cars a reality, but I don't want to dive too deeply into that. One thing that's obvious is that self-driving cars have already been promised to the public (Waymo claims to have launched last year but it's really just a private beta with a handful of users). Cruise is supposed to launch a public service this year in SF. Zoox is supposed to do the same in 2020 (and in custom built cars, no less). And there's no doubt that machine learning plays a critical role in the advancement of self-driving cars.
Now if you talk to the actual researchers/engineers when they're not afraid of hiding behind their NDAs (basically a friend at a bar), these people will talk about the challenges and how much work there is to do and so on. But the company's public stance, and the many billions of dollars chasing this area, is that it's just around the corner.
The point is, there is a lot of hype and it's easy to understand why. Elon Musk is happy to proclaim that Tesla is already selling the hardware for full self-driving vehicles, and it's just the regulators holding him back (which is absurd, to put it kindly). This is the same person saying AI is a bigger threat than nuclear weapons. So arguably the largest tech celebrity isn't even talking about when, he's saying it's here and we need to be prepared. This is the kind of person shaping the public's opinion.
But the money chasing these investments are mostly from private VC firms and tech companies, not public research grants. Once investors realize AI just isn't good enough for self-driving cars right now they won't want to keep throwing billions of dollars behind it. And that's when we arrive at a winter. Even Alphabet is starting to get weird about raising its stock price, so not even the tech giants are immune from this.
There are many other verticals that follow the same story, I was just using the most obvious example of an industry that has billions flowing into it despite nobody having a real product on the promise of advancements in AI and "the time is now!" But eventually (my guess is next year, but I'm old enough to know it's impossible to time these things) the hype will come crashing down to reality and that is when funding will dry up.
I was at a local government plenary session at Chengdu (long story) a couple of year back. The lead speaker kept waxing about how China invented and contributed AI to the world.
I mentioned some points about about China's AI contributions/influence :
0. Algorithms -> The OGs as celebrated by this Turing Award (Canadian, but mostly led by American universities. On thing I did mention was due to the sheer factory production of Chinese PhDs, there is a lot of stuff arxiv from China )
China has some equivalent for all of them, ie. PaddlePaddle by Baidu, AliCloud, etc. but none of them have the reach, influence or domination of the American counterparts.
Suffice to say that my points weren't taken too well and have been disinvited since then.
Do you really think China has done anything on the level of these three? Basically every new neural network architecture in recent memory is underpinned by Hinton's work on back-propagation.
China has done some fantastic work in ML, but I think the award was a long time coming for these three.
This is really quite silly. You have no idea how much of recent AI innovations and breakthroughs are coming from China. Many of the top AI conferences these days have quarter to half of the accepted papers originating from China. I am not even counting tremendous amount of contribution in all of above from the first generation Chinese living outside China.
I think the lead speaker was full of hot air, but let's not fall into the trap of overestimating the changes in 2 years and underestimating the changes in 10, assuming the US government doesn't have a response to china's AI initiatives.
I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.
Schmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding.
LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolution are starting to replace them.
Ironically Schmidhuber's lab also came up with a more general version of bypassing http://arxiv.org/abs/1505.00387 to train deep networks (with similar performance), of which the popular ResNet is a special case.
Putting aside the whole Schmidhuber debate - where are people getting this idea that causal convolutions are anywhere near the prominence of RNNS/LSTMs?
As far as I'm aware, causal convolutions were used in WaveNet (and subsequent models) and a small number of NLP applications. Meanwhile, LSTM-based models are used in just about every NLP paper, and at least a baseline in the newer ones more dominated by Transformers.
That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM). Instead of reading a few cherry-picked results from a model with extremely tuned hyperparameters, pick any random set of tasks and experiment yourself. Based on my experience, LSTM always performs better than all unless you brutely search over hundreds of hyperparameters configs.
> That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM).
But that's exactly the point of the Transformer model, with a paper aptly titled "Attention is all you need" [1]. And the Bert architecture, based in this idea, seems to be doing well. And they claim to be bery flexible, too[2].
Maybe that's what you meant with "unless you brutely search over hundreds of hyperparameters configs", but then again, isn't that what NNs are about anyway?
The success of Transformers aside, I'm not sure you should be relying on model titles for anything, lest we forget papers like "One Model To Learn Them All" [1].
Whoa. Personal attacks will get you banned here. We've warned you about this more than once before. Please review https://news.ycombinator.com/newsguidelines.html and follow the rules when posting here. When you do, don't miss this one, since it should have changed your assumption about the GP:
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
Just a note, I did not find it any more rude than other comments that get a free pass. Commenting that more familiarity with the literature is necessary can be a very factual claim.
It was an obvious violation of HN's rules. If you see other comments that violate the rules going unmoderated, the likeliest explanation is that we didn't see them; we don't come close to seeing everything that gets posted. In that case the thing to do is to flag the comment (described at https://news.ycombinator.com/newsfaq.html). In egregious cases, you can email hn@ycombinator.com as well.
I'm not saying that Schmidhuber is not important pioneer in the field.
When you put researchers or groups into order of importance, the work from this trio comes before others, including Schmidhuber. The Turing Award is given to those at the top and it's not inclusive.
If I could give Turing Award for someone in the field who has not received it yet, I would give it to Vladimir Vapnik.
>If I could give Turing Award for someone in the field who has not received it yet, I would give it to Vladimir Vapnik.
So would I, but I don't think that will happen. He's is pretty much an antithesis to everything Deep Learning is about. Theory-first over trial-and-error, math over intuition, small datasets over big data, advances in understanding vs advances in results.
I haven't seen a single discussion of his paper on combining classifiers[1] anywhere on the web.
If LSTM is up there, maybe Transformer authors should also be listed somewhere? I mean LSTM is fine and all, IIRC, it only becomes popular after one of Hinton's students put it to work? And that is like in 2013. Similar tales to the CNN as well.
Hinton/Lecun pioneered early BP research, which is significant and fundamental. Bengio has attention mechanism/GAN, and in early days greedy layer wise pretraining for RBM (which isn't something end up today, but pertaining is a thing back in pre 2010 era).
Not saying LSTM isn't significant, but his achievements aren't quite there with those 3, looking closer.
Yes, and the vast, vast majority of important papers by the three winners were co-authored in the same way with supervisors, graduate students or other collaborators.
I encourage downvoters to read the article below for a perspective on how LeCun, Hinton and Bengio may have unfairly excluded a wealth of prior work in their version of deep learning history:
Extremely pleased to see the weird feud with Schmidhuber sparking up again, completely pointless feuds like that are very entertaining, especially since all involved figures have done good work and don't seem to be sabotaging each other's students. It's like pro wrestling for nerds.
> On 30 September 2012, a convolutional neural network (CNN) called AlexNet achieved a top-5 error of 15.3% in the ImageNet 2012 Challenge, more than 10.8 percentage points lower than that of the runner up. This was made feasible due to the utilization of Graphics processing units (GPUs) during training, an essential ingredient of the deep learning revolution. According to The Economist, "Suddenly people started to pay attention, not just within the AI community but across the technology industry as a whole."
Congrats! Absolutely well deserved. Simons' Foundations of Deep Learning seminar this Summer 2019 will seek to illumine some of the issues around reproducibility, failure modes, etc
Companies should jointly give CIFAR a big fat award too. :-D Without CIFAR's 10 million dollars back in 2003 to the research groups of 15 people, led by Hinton, the research groups may have dissolved due to lack of funding, and our history would be different.
Yeah that might be a better name. Sometimes it's called a "computation graph", since that's exactly what it really is. A network of nodes computing weights and derivatives based off inputs and/or outputs.
Tbqh schmidhuber absolutely deserves a spot here. Not just for LSTMs, but also predictability minimization, various interesting intrinsic motivation concepts etc.
Well deserved. I was thinking maybe it's a bit too soon (the Turing Award isn't exactly timely awarded) but remembered it's 2019...
If you wanted a first-rate CS education, you could do a lot worse than to go through the winners of the Turing Award[0] and review their seminal works. I've only sampled maybe half of them but every time I pick one I learn something interesting.
It's interesting to me that most comments are confusing Deep Learning with the whole field of AI in general, when it is actually a subset of Machine Learning.
Well deserved anyhow!
I somehow associate Hinton with Terry Sejnowski more than his later collaborators. They are all pioneers and this is well deserved regardless, even late!
Do you think (HN) that this incentive boost will mean more people who're working on fringe risky ideas will get the energy and persistence to keep going? Will we see a bump in 10 or so years of innovations coming from people who would have otherwise given up (had this award not been given)? Hard to measure of course, but what's your hunch?
If anything it's demotivating. Other people that have also contributed greatly were just ignored by this award. While those that received the award were mostly represented by big corps Facebook(LeCun) and Google(Hinton) -- even though these three are deserving as well, especially Hinton. Awards like this are quite stupid. The award isn't even based on a single breakthrough, but on their "collective achievement." This was just a PR thing. Who does science for awards anyway? I thought that was an athlete thing.
There's a Youtube channel called something like 'AI Channel' that has relevant talks. LeCun did a big one a few months ago. Hinton has had at least one Google Talk.
as a non-specialist, it seems like several important and distinct areas of inquiry are just lumped together in the celebration of a "winner" for ML
Text, language, human chat
Image recognition from blurry, multiple views, multiple lighting conditions photos
formal patterns with large numbers of variations
These are not at all the same, yet the praise seems to want to declare "the best" and "beating competitors" .. why is something so multi-faceted, reduced to the logic of a sports event ?
The achievement is mainly about the fundamental techniques they developed, not the application domain. The techniques have been applied to many things since, hence the large impact.
i want to get on the hype machine and holler inaninties while i shoot my revolver into the ceiling.
we must glorify the efforts of a arbitrarily chosen single individual, in the hope of someday becoming that lauded singleton, the one allowed by decree to piss on the heads of those fools below who didn't win the race.
how much is changed since those geniuses started to work in this particular field? Me coming from web full-stack dev and see how things change fast and I find myself jumping from A way to B way very quick these days. I just need to know your perception of how fast thing goes and changes on ML.
You change frameworks and tools pretty frequently but they are still all built around the same building blocks: html, css, javascript etc.
It's pretty much the same in ML. These people built the building blocks that we still use in ML everyday (e.g. backpropagation, ConvNets etc). But the fine-tuning, packaging and tooling of these techniques also changes all the time and it can be hard to keep up.
Having moved from full-stack web to ML I have a similar feeling about the pace of things.
I have a browser extension that replaces the phrase "Artificial Intelligence" with the phrase "A bunch of if statements". This might have been one of the top results.
I'm also quite dismissive of AI overuse. My preferred substitution is s/Machine Learning/Machine Guessing.
However, these are 3 people who overcame a lot of nay-sayers to prove some something could actually work. I remember back in university the lecturer said that neural networks couldn't scale because of the vanishing gradient problem.
It was actually very easy to be critical, you want fit a massive number of parameters which are not-very-orthoganal to an under constrained problem? Sounds pretty dumb to me!
I think these 3 deserve recognition for their tenacity and the new world of "under defined gradient decent", sometimes called DL, which they opened up.
The description available on the ACM website right now doesn't mention AI: "For conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing." [0]
As it should be. The whole AI buzz misleads people into thinking we are close to solving things we actually are not. Like upload filters capable of differentiating fair use and copyright infringement.
You guys need to get over it. Intelligence is not a well defined term. AI is fine do describe something that works "humanlike" even though we don't kno why. I would dare you to give a definition of "AGI"
It saddened me when Boyle and Smith won the Nobel prize in Physics for inventing CCD which powers digital cameras and is of no use other than surveillance by big brother. Don't get me started about Alexander Graham Bell /s
Science and technology are generally agnostic to their usage, and will be used to both positive and negative ends whose magnitude is proportional to the power of the underlying technology.*
AI has the potential for massive positive effects as well as massive negative effects.
What saddens me is that we have systems and incentives in place which encourage governments and corporations to use powerful technologies to negative ends.
* Obviously there is room to quibble about the positive/negative balance of individual technologies, e.g. gunpowder, bio-weapons, etc., however I am sure a circumspect analysis could find positives even in things developed primarily to improve the efficiency of war.
are you serious? there is no way to do research on massive amounts of data at the public level anymore. if you want to change things its in the private. + money is also good
I didn't realize the internet or cars were created by the employees of advertising corporations for the sole purpose of furthering their surveillance and user manipulation capabilities.
Also, my comment was about people being given awards, not people inventing things.
I didn't realize the internet or cars were created by the employees of advertising corporations for the sole purpose of furthering their surveillance and user manipulation capabilities.
Seriously man? Do you think you might be being just a little bit hyperbolic there? I mean, it's pretty darn clear that Deep Learning isn't "created ... for the sole purpose of furthering ... surveillance and user manipulation capabilities" when almost all of the papers on the topic are freely available on arXiv, and source code for nearly every important implementation is available as Open Source, usually in multiple languages, using multiple different frameworks (which are almost all Open Source as well). I mean, sure, Google use Deep Learning to display ads... great. But you, or me, or anybody else, can also use the same techniques for our own (presumably more noble) ends.
I have the feeling this got so many upvotes because people think this is related to the turing test. And that we have a chatbot now that can trick the average person into thinking it's a human.
It's not.
It's about some incremental progress in making neuronal networks classify stuff.
But it's not about "well known" anyhow. It's about significance. A turing test winning chat bot would be a much bigger breakthrough then an incremental improvement in image and text classification.
Is this a meme? No they have not. Sure they are a popular hardware provider, but NN have been around before they were even a company. I think Fukushima deserves a spot here too, before giving it to Nvidia, but there are so many people that have contributed to NN, so I guess those in the spotlight take the glory if it must be taken.
He's credited with first introducing the CNN architecture (Neocognitron) and a way to train it with unsupervised learning, inspired by the neurological research of Hubel and Wiesel and his earlier work in training neural networks with unsupervised learning (backprop is now used instead). Probably one of the most famous neural network papers. https://www.rctn.org/bruno/public/papers/Fukushima1980.pdf
They don't deserve science award because they didn't do science or pioneer anything. They just responded to demand. They have been awarded with money.
It was the high performance scientific computing community that started to use graphics cards to perform matrix computations in the early 2000's. The term GPGPU term was invented. GPU programming pain in the ass in early 2000's. Nvidia saw some markets in GPGPU's for scientific computing. First release of CUDA was 2007.
> They don't deserve science award because they didn't do science or pioneer anything.
NVidia didn't do science? It depends on what you call science (math and computer science are not "science" according to a very strict definition involving falsifiability). But in any case, an enormous amount of research went into the development of GPUs. It's not just engineering work. Lots of PhD (students) contributed.
True, but neural networks existed for a long time. The hardware is what made NNs actually practical. Recent theoretical advances have been only incremental. For example, CNNs are just evolutionary since image recognition has used convolutions long before NNs were practical, and CNNs are just a straightforward translation of existing techniques to NNs.
Sure theoretical advances may be evolutionary, but they resulted in exponential reduction of parameter space and sample complexity. These advances outpace Moore's law by a large margin.
Hardware served as a catalyst, but it was not a necessity.
You could instantly see the results he presents were way better than what was state of the art at that time. Amazing.
---
[0] Grant Sanderson (3Blue1Brown) started a youtube-series covering Neural Networks (4 episodes, so far) that helps gain an intuitive grasp on the topic: https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_6700...
[1] https://en.m.wikipedia.org/wiki/MNIST_database
[2] https://en.m.wikipedia.org/wiki/AlexNet