Uncertainty is a difficult concept to formalize because everything is connected to everything else in principle -- that is, relative to a joint probability distribution. One of the reasons why expert systems got stuck was the lack of a complete and correct theory for how to reason with uncertainty.
A medical diagnosis system like MYCIN [1] is fundamentally at war with uncertainty, but systems like that used half-baked ad hoc methods of modeling it.
It's a problem in machine learning because in principle you're looking at a high dimensional space where the joint probability distribution has a huge volume that you can't sample comprehensively. Insofar as machine learning, particularly deep learning, models do a good job of guessing the joint probability distribution without a comprehensive sample.
A medical diagnosis system like MYCIN [1] is fundamentally at war with uncertainty, but systems like that used half-baked ad hoc methods of modeling it.
It's a problem in machine learning because in principle you're looking at a high dimensional space where the joint probability distribution has a huge volume that you can't sample comprehensively. Insofar as machine learning, particularly deep learning, models do a good job of guessing the joint probability distribution without a comprehensive sample.
[1] https://en.wikipedia.org/wiki/Mycin