This is a great point. There is a general lack of understanding about what it means for models to be interpretable & explainable. These words get thrown around often by people who don't understand the definition, and also the trade off with accuracy.

Some papers i found interesting on the subject:

https://arxiv.org/abs/1606.03490

https://arxiv.org/abs/1707.03886

https://arxiv.org/abs/1806.07552

https://arxiv.org/abs/1702.08608 (i found this was a good sumary of the issues)