He also mentions sparsely activating only the neurons that matter, which they explore in https://arxiv.org/pdf/1701.06538.pdf

Personally I didn't find it very satisfying, I imagine something more fundamental and self referential.