Hello! Thanks for pointing out that my CV is out of date. You're right that none of my work over the last two years is listed there, and I should probably fix that.
It feels a bit awkward to chime in, but it feels a bit important because I think there's a way in which the narrative is a little off (although I'm very grateful for everyone's kind comments):
Working at Google wasn't a trade off where I got financial support in exchange for not publishing. Rather, working at Google was amazing enabler, which supported me doing work I believed advanced important public interests for five years. That actually includes the blog posts being discussed here, which I wrote while an intern on Brain and received feedback from many researchers there.
To be concrete, some of the things I did while at Google included:
* Writing expository blog posts explaining important ideas in machine learning (eg. this blog post).
* A five year thread of research on neural network interpretability. How humans can understand how neural networks make decisions? (eg. DeepDream, Feature Vis, Lucid, Building Blocks)
* Cofoudning a scientific journal for machine learning that values exposition and allows interactive diagrams.
* Working on early TensorFlow and writing some of the initial tutorials on it.
* Articulating concrete safety concerns about modern machine learning systems, organizing a cross-instutional paper on it, and later representing Google on the Partnership on AI's Safety-Critical AI working group.
* Designing and teaching the introductory course for the Google Brain/AI Residency.
* Miscellaneous ML research, all published in open access venues.
That's basically the majority of what I did at Google. I always felt empowered to work on things that I thought were important for the world, and in many cases felt like had leverage I wouldn't have had individually. I think that was exceptionally generous of Google, and also speaks a lot to the environment Jeff created for those of us on Brain. (I realize Google is a large company, and other people's experience may be very different.)
I do think you're right that the number of short expository blog posts I wrote declined, especially after my first two years at Google. A big part of that is that I focused more on a small number of more ambitious projects (eg. Distill, Building Blocks). Another big part is that I started doing more non-individual work: teaching, mentoring, editing, debugging social issues.
To be clear, I actually left Google last fall. I now work at OpenAI where I lead the new Clarity team, which works neural network interpretability -- basically, can we take a trained neural network and turn it into something like code that a human can understand? Leading a team means that I do even less individual research and writing. Sometimes I miss being an individual contributor, but I think it's the right call. I get to build an environment where others can focus on a kind of research I think is really important for the world and teach and support them.
I feel like someone who would write an article like "Research Debt" is someone who truly gets it. Someone who sees the enormous upside potential that can be unlocked through just investing in quality.
I'm deeply interested in these topics, but I'm not so good at them. I have gotten much further than I ever would because of efforts like these.
We're still ramping up. So far, our only publication has been co-authoring Activation Atlases with our wonderful colleagues at Google (https://distill.pub/2019/activation-atlas/).
One thing we're thinking a lot about how you can transition from "what" a neural network represents to "how does it mechanistically do that"? I hope we'll publish on that in the next couple of months.
By the way, while I'm flattered by the esteem it implies, I'd generally rather people not refer to us as the "Olah Team", just as you wouldn't refer Google Brain as "Jeff Dean Research" or MILA as "Yoshua Bengio Institute" or OpenAI as "Sutskever AI", etc. I think academic culture of branding groups after the PI is kind of unhealthy.
My teammates are doing the hard work and I see my position as a team lead as just to serve and support them. :)
It feels a bit awkward to chime in, but it feels a bit important because I think there's a way in which the narrative is a little off (although I'm very grateful for everyone's kind comments):
Working at Google wasn't a trade off where I got financial support in exchange for not publishing. Rather, working at Google was amazing enabler, which supported me doing work I believed advanced important public interests for five years. That actually includes the blog posts being discussed here, which I wrote while an intern on Brain and received feedback from many researchers there.
To be concrete, some of the things I did while at Google included:
* Writing expository blog posts explaining important ideas in machine learning (eg. this blog post).
* A five year thread of research on neural network interpretability. How humans can understand how neural networks make decisions? (eg. DeepDream, Feature Vis, Lucid, Building Blocks)
* Cofoudning a scientific journal for machine learning that values exposition and allows interactive diagrams.
* Working on early TensorFlow and writing some of the initial tutorials on it.
* Articulating concrete safety concerns about modern machine learning systems, organizing a cross-instutional paper on it, and later representing Google on the Partnership on AI's Safety-Critical AI working group.
* Designing and teaching the introductory course for the Google Brain/AI Residency.
* Miscellaneous ML research, all published in open access venues.
That's basically the majority of what I did at Google. I always felt empowered to work on things that I thought were important for the world, and in many cases felt like had leverage I wouldn't have had individually. I think that was exceptionally generous of Google, and also speaks a lot to the environment Jeff created for those of us on Brain. (I realize Google is a large company, and other people's experience may be very different.)
I do think you're right that the number of short expository blog posts I wrote declined, especially after my first two years at Google. A big part of that is that I focused more on a small number of more ambitious projects (eg. Distill, Building Blocks). Another big part is that I started doing more non-individual work: teaching, mentoring, editing, debugging social issues.
To be clear, I actually left Google last fall. I now work at OpenAI where I lead the new Clarity team, which works neural network interpretability -- basically, can we take a trained neural network and turn it into something like code that a human can understand? Leading a team means that I do even less individual research and writing. Sometimes I miss being an individual contributor, but I think it's the right call. I get to build an environment where others can focus on a kind of research I think is really important for the world and teach and support them.