Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
DeepMind’s work in 2016: a round-up (deepmind.com)
212 points by jonbaer on Jan 3, 2017 | hide | past | favorite | 55 comments


I can't believe this one passed under my radar:

https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...

> Our partnership with Google’s data centre team used AlphaGo-like techniques to discover creative new methods of managing cooling, leading to a remarkable 15% improvement in the buildings’ energy efficiency.

Which itself is an understatement of the achievement:

> Our machine learning system was able to consistently achieve a 40 percent reduction in the amount of energy used for cooling, which equates to a 15 percent reduction in overall PUE overhead after accounting for electrical losses and other non-cooling inefficiencies. It also produced the lowest PUE the site had ever seen.

Basically, they built an AI that was able to tune the "large industrial equipment such as pumps, chillers and cooling towers" to react to the dynamic, nonlinear interactions that vary within and between datacenters (weather, utilization, etc).

They also describe this AI as "General":

> Because the algorithm is a general-purpose framework to understand complex dynamics, we plan to apply this to other challenges in the data centre environment and beyond in the coming months.

They seem to imply that this technique could make almost any industrial process more efficient with minimal oversight/training/customization.


I remember reading the blog post. Did they ever follow up with a research paper explaining what they actually did?


Their past paper on this: https://www.google.com/about/datacenters/efficiency/internal...

In the blog post in July 20 2016 they said "We are planning to roll out this system more broadly and will share how we did it in an upcoming publication" https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...


Sadly, not yet. Might be too secret sauce even for Google.


Nonlinear control with neural networks has existed for a long time, though, it's not exactly new. Is there reason to think they did something profoundly new in that specific instance?


Probably the use of a DNC for industrial application is novel.


Yes. When this came first up, someone on /r/machinelearning scoffed that using NNs to control fans/coolers/power systems to optimize was trivial and obvious and had been done for decades. When I challenged him for a ref, he produced a PDF which listed dozens of industrial applications for various nonlinear NN algorithms... none of which were actually autonomously controlling anything and most were predictive.


It's not secret sauce, the algos are well known, read carefully, the guy who proposed it was an ML outsider. And besides, they stated they plan to release it.


Lol, no. It's just the scale of the use that makes it valuable. Google datacenter cooling problems only exist inside of Google data centers, and the control knobs therein are 1. measurable AND measured 2. deterministic 3. simple.

They're innovating in a problem space that only exists at Google, Microsoft, and Amazon.

This is the boringest of sauces.


And you gotta realize that given the massive power usage at Google, this is huge, both for the environment and for Google's financials.

In 2014, their data centers used 4,402,836 MWh. Even at 25$ per MWh, that's already $100B, so this little project basically saved them around $40B...


Its 100 millon not 100 billion


> In its ability to identify and share new insights about one of the most contemplated games of all time, AlphaGo offers a promising sign of the value AI may one day provide, and we're looking forward to playing more games in 2017.

What I find fascinating is how different AlphaGo's impact was from the impact of early chess engines. Once chess engines became decent (not even good—before even Deep Blue), they identified tons of inaccuracies in published chess literature. These were missed tactics, hard-to-see moves, requiring only relatively shallow calculation but a computer's precision. These inaccuracies were found even in classical annotations of well-known chess games, as well as standard books about openings. I believe John Nunn was known for this kind of work.

AlphaGo hasn't achieved nearly the same impact. Have they even tried to identify the same types of inaccuracies in classical Go books? Can you imagine how absolutely cool it would be for a go engine to find errors in Invincible? Maybe they tried, but didn't find any inaccuracies, so now this negative result is sitting in one of their file drawers? I really wish they were more active with this sort of thing.


Once chess engines became decent (not even good—before even Deep Blue), they identified tons of inaccuracies in published chess literature.

Within one year of their invention?


Well, AlphaGo beat Lee Sedol, so I think it wouldn't be fair to compare timelines so directly. With chess engines, the compute power just wasn't there in the beginning, but it's all already there for go. Also, speaking as an optimist, yeah! why not? To be honest, my suspicion is that they aren't trying.


I suspect this is a visibility problem. Go is a more popular game in the East I bet there are a few grad student level problems doing just what you described at Tokyo U and other prestigious Schools far from the West.


Good point, I forgot about DeepZenGo. But it's very low-hanging fruit, though, which is the main reason why I'm surprised they haven't already done this.


Are there any open source or widely available Go engines that are even remotely as strong as AlphaGo (which I think we can safely assume plays beyond the 9p level, i.e., at superhuman strength)? If the only seriously strong engine is completely tied down behind the closed doors of a company that has largely already moved on to other challenges, I think that's the issue right there -- or at least a large part of it.


I think the strongest commercial engines are Zen and CrazyStone. Zen has certainly become pro-strength on fairly modest hardware (it had an exciting BO3 against a Japanese 9-dan, which it lost 1-2), but the strong version isn't available to the public yet, I think.

Interestingly, for the past 3 or 4 days there's been a mystery Go Bot playing on 2 servers calling itself "Master" which is currently about 50-0 (!) against professional opponents, many at the very highest level. Nobody knows its identity yet. It's stronger than the commercial engines, and its presence must be some sort of stunt or test by someone.

http://lifein19x19.com/forum/viewtopic.php?f=10&t=13913



It's 58-0 now. They've been playing 10 games per day since December 29.

Diagrams with moves at https://tieba.baidu.com/p/4922688212?pn=150&

Google translate helps with the comments but they don't add much to the diagrams.

They are playing fast games with 3 time periods of 30 seconds per move. It means that one can exceed the 30 seconds limit and move on to the next period. When the limit is exceeded in the last period the game is over.

Fast games usually increase the chances of the stronger player if the difference is large.

Edit: 59-0 just after I posted this.


I think the next strongest engine is Zen ( http://senseis.xmp.net/?ZenGoProgram ) which plays at amateur 9 dan level. CrazyStone is strong too.

Strongest open source engine is Pachi, which has achieved an amateur 4 dan ranking on the KGS go server. Fuego is almost as good.


Go is different than chess. There is no huge blunders to be found in well-known games. What Alphago can find are creative moves that happen to be very slightly better than "normal" ones.

Since a week, I hear news about a bot named "Master" that may be an evolution (or a concurrent clone) of Alphago.

You can study the games: http://www.dgob.de/yabbse/index.php?topic=6381.msg208264#msg...

There are speculations that the next game between Alphago and best players would be an handicap game.

IMHO Alphago is starting a revolution in the fuseki theory that is bigger than fixing a couple of broken openings.


Deepmind is more focused on general AI (AGI) than the game of Go. The AGI community has always considered beating a Go champion to be a significant milestone towards AGI, and it was expected to be years in the future. Now that Deepmind won that battle, there's little PR benefit - or benefit to AGI progress - to focusing on go.

I don't mean to slight the game of Go, but it is a perfect information game which could make it non-central to AGI research.


I'm excited for WaveNet to get faster/more accessible. I'm not very technically inclined when it comes to downloading things of github and hacking them together.

My goal is audio books - I'd love to hear them read by my favorite movie characters.


I can't wait to have Peter (Geoffrey Francis) Jones read me all Wikipedia articles.


They had a vast presence on EWRL, it's really amazing how many effort goes into research. Really inspiring.


> We’re still a young company early in our mission...

What? Why don't they consider themselves part of Google?


I don't know why, but it's very clear they consider themselves somewhat independent of Google.

The most clear sign I have seen is that Partnership on AI site has both Google logo and DeepMind logo.

https://www.partnershiponai.org/


Probably by design.

I assume Google wants to have them doing their own thing instead of just borg-similating it. Also they have Google brain, which is doing similar things


What indeterminate games can AI can beat humans at?

Backgammon - AI usually beats the best humans

Ms. Pac Man - AI loses to almost any human


AI tends to perform poorly for many real-time games because they constitute many, many more "moves" than a game like backgammon. A backgammon game will typically take < 100 moves. For a game like Ms. Pac Man the AI will have to make that many decisions in a timespan of about four seconds. When you train a reinforcement learning algorithm, the AI has to update the parameters that were used to make several thousand decisions, and the only information it has to go on is its score at the end of the game. (This is known as the credit assignment problem --- of all the decisions you made in the game, only a few probably had a substantial impact on the outcome. How do you decide which ones were the important ones?)

A game like go is hard because there are many possible moves every turn. But the games themselves are not very many turns, so it is easier to find the important moves, because in some sense all the moves were more important. A game like Ms. Pac Man is hard because, while you only have 5 moves each turn, the games themselves are many, many turns, and it is very difficult for the NNs to learn these long-term dependencies.


Since a child could easily outperform the best AI here, doesn't it imply we need a different approach?

Otherwise we need 4 orders of magnitude more compute, and it's just hard to believe their is no better way.


No, just means that someone needs to bother to make an AI for Ms. Pacman.

Handling random ghost movements is only trivially confounding for AI learning. Low value, low yield.


It's true that someone could pretty easily hand-craft an AI that plays Ms. Pac-Man pretty well. But that's not really the point. Because after all that work, you still only end up with a computer program that can play Ms. Pac-Man. It won't play Pong, it won't play Doom, it won't even be able to play regular Pac-Man! The goal is to build a general system that can learn how to play specific games on its own. So, there is potentially high value to developing a Ms. Pac-Man AI, but only if it's not really a Ms. Pac-Man AI, but actually a more general AI, for which playing Ms. Pac-Man is a specific instance.

I agree with the parent comment that some sort of new approach is probably needed. The main issue with reinforcement learning is just that the algorithm gets too little feedback. This ends up discarding an enormous amount of data on each run (or, more accurately, learning from that data extremely inefficiently). There's a huge amount of data present, but the learning from that data has to be unsupervised. A child can do it easily --- they only have to play Ms. Pac-Man once or twice to get the hang of it. Computers currently cannot. Getting them to do that will definitely require something different, but no one is quite sure what.


How much more complex is Ms. Pac-Man when compared to Atari 2600 games? Deep Minds DQN did master (as in better then any human can play them) a whole bunch (49) of 2600 games a while back

https://deepmind.com/research/dqn/

edit: Checking wikipedia looks like it isn't playing all games it can play at superhuman levels

> For most games (Space Invaders, Ms Pacman, Q*Bert for example), DeepMind plays below the current World Record

https://en.wikipedia.org/wiki/DeepMind#Deep_reinforcement_le...


DM did terrible at Ms. Pac Man, let alone below the current world record is an understatement.

DM scored less than 30,000. Human record is over 900,000.


>someone could pretty easily hand-craft an AI that plays Ms. Pac-Man pretty well

Not true. Many people have also tried handcrafting as well and only scored around 30,000 which is a low score.


I think you are underestimating the difficulty. It's not like DM was the first to try.

Multiple papers have been written on failed attempts. Dozens of other developers have tried.

It's an open problem.


It's a boring tautology. Arcade games require "clever" (read: not generalizable) solutions to optimize for speed in decision making.

"Learning" AI requires the broadest feasible search space for any solution and successful designs require clusters of machines. Clusters of machines have network latency and coordination overhead.

This is the same latency vs throughput design decision that every system must make, and its not impressive when a throughout oriented system struggles with being latency sensitive.


It may be boring because you don't realize it's not a tautology.

You are wrong to separate video games from generalized AI. If you read the paper I posted above it explains part of this active area of research,


Conventional AI can clean up at Pacman if it can do tree search on the game state, source: CS188 out of Berkeley.

http://ai.berkeley.edu/lecture_videos.html


Pac Man is easy. Ms. Pacman has been tried and failed by many people who know CS188 material.


Do these attempts make use of the documented idioscynracies of Ms Pacman?

http://gameinternals.com/post/2072558330/understanding-pac-m...


Note that most of recent work in reinforcement learning focus on creating a model which works on all or most of Atari games without using any game-specific knowledge. If researchers were to focus on only Ms. Pac Man, and ignore the "no game-specific tuning" rule, then Ms. Pac Man, and most Atari games would be solved to near perfection. The challenge that the Atari suite provides is not their individual difficulty, but the difficulty imposed by their diversity, i.e. finding a general algorithm which works well on all Atari games is hard.


I don't know why so many people are suggesting this is easy.

Many other attempts has been made targeted specifically against Ms. Pac Man and none have come anywhere near beating an average human player.


What makes Ms. Pac Man so hard for AI?


Its indeterminate because the ghosts turn in random directions with no preserve patterns. Notice Go, Chess, etc all are determinant.

This paper gives an overview of the background: http://www.cse.unsw.edu.au/~mit/Papers/AAAI10a.pdf


I'm not sure it's hard in general but deepmind's program didn't do well because it couldn't plan ahead https://www.technologyreview.com/s/535446/googles-ai-masters...


It's also inferring the game state from looking at the screen rather than being spoonfed the data.

One could conceivably perform a depth-limited search on the actual game state if it were available and then use an AlphaGo-like DNN to predict what a deeper search would find, no?


Dunno really. A human would watch the ghosts behaviour and guess their likely future behaviours based on that. I'm not sure if the software gets that, as it were, or how you'd tweak it to do so.


Does anyone know of any other companies that are similar to DeepMind, but based in SF?


OpenAI (https://openai.com/). OpenAI's research vision is at a similarly ambitious scale as DeepMind. They're also relatively more open in their research, and known to release papers, code, and models quickly. Currently, they are not as big though, more like how DeepMind was immediately after publishing their first Atari work in 2013. Give them time, and I expect them to become a very formidable opponent to DeepMind.


As a bonus, OpenAI is a nonprofit that always allows publishing. They also have some of the top names in DL: Ian Goodfellow, Ilya Sutskever and Zaremba. In DRL (deepmind's arena), they have Abbeel and Schulman, both absolute houses.


The Google Brain (g.co/brain) team has people in SF. We are part of the same company as DeepMind so maybe this doesn't quite answer your question. ^_^

We are mostly in SF and Mountain View, but we also have people in a few other locations. Right now, SF and Mountain View are the largest.

Disclosure: I work for Google on the Brain team.


OpenAI? (It's a non-profit, not sure if you personally find that truly disqualifying)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: