I don't think anyone has mentioned Bayesian Neural Nets (I forget the exact term). Sure, the paradigm adds an order of magnitude overhead (at least - and that's why I've never seen it used in the industry), but you can bolt it on to existing architectures.
The basic idea is that besides the probabilities, the network also spits out confidence (IIRC based on how out-of-distribution the input is). There's been a ton of work on getting confidence values out of existing neural nets without as much overhead, but I've never seen those approaches replicate in the industry.
I would imagine that to propagate any confidence value through the system you'd need to have priors for the confidence of correctness for all data in your training set. (and those priors change over time)
The basic idea is that besides the probabilities, the network also spits out confidence (IIRC based on how out-of-distribution the input is). There's been a ton of work on getting confidence values out of existing neural nets without as much overhead, but I've never seen those approaches replicate in the industry.