You're right. In order to be a big problem, the error needs to be non-cancelling and inseparable.

That makes me wonder: if you label good data, and generate data with the good label, how much benefit do you get from also training on okay data?