Exactly. If we are going to hypothesize about time travelers then we have to accept that any significant reason to time travel has already happened. Futhermore it has happened more times than we are aware of to create our specific timeline.

Maybe Hitler was a time traveler himself.