Yeah, but now you need to know the commit hash for all the branches. Basically, you need a form of backup. But this problem presumes you don't have much of a backup.
Oh sure, but at the end of the day some developer or group of developers is in charge of those repositories, and considered a trusted person (or effectively is).
Between them and the large number of developers who would have copies of the repo, reaching consensus on what the "true" repo was - while not easy - could be done in a secure fashion due to the hashes. You wouldn't have people declaring "no it's totally it" and not being able to verify.