This is such an important point that I think escapes a lot of people, both on the pro-AI and anti-AI side.
Although everyone probably wishes otherwise, there isn't really a hard objective line for determining if something is infringing or not. And that's actually a good thing! But it means that, in the end, it's up to a judge to weigh a bunch of different, rather fuzzy, factors.
Law is not a static entity, it's morphing and evolving to suit our ideals today. Learning wasn't an issue until it is.
We could very well distinguish between machine learning and human learning although they're no different from each other in principle.
And although, we can't say that ML is plagiarism, at scale it breaches our other moral principles like privacy and individual identity.
Let's say in 10 years from now Google'll train ML on (close to) all the data available in the world, be it text, visual or audio. Would they be able to prompt this AI: "You are George Wilkinson from Colorado ** st. 23". How close will it be from real George on whose data it was trained on?
We can't tune our human learning to this level of precision, but it is only a matter of time for the machine.
The situation you describe is exactly that which has already existed for the last two decades with Google using that data for their search engine.
If it's public it can be read.
Public book libraries have been free to read since centuries, and yet no single human has ever read even 1% of them. We can't ignore that different scales = different consequences. (and to put them together the authors were paid)
Although everyone probably wishes otherwise, there isn't really a hard objective line for determining if something is infringing or not. And that's actually a good thing! But it means that, in the end, it's up to a judge to weigh a bunch of different, rather fuzzy, factors.