A backup of a corrupt repository would have been just as corrupt though.
This is the big thing I can't figure out what people are not understanding. git does consistency checking for you already, tar|rsync|etc. don't, so it makes sense to take advantage of that.
What we had was an instance of some of the underlying data becoming corrupt on the filesystem (with indications of that starting on Feb 22!). The big mistake was considering the source repositories as consistent and canonical at the remote anongit end, but the data would have been just as corrupt if we had scp'ed the repos from git.kde.org to the anongit mirrors around the world, since we would have bypassed git's internal checking in that way.
Is it safe to rsync a running mysql database at random times, or are you supposed to use mysql-provided tools to perform a backup?
OK, but what stops them from daily performing a mirror clone, checking it for consistency, then backing that up? As mentioned in the linked update, 30 complete backups would consume only 900GB, so you could keep weeks of daily backups, plus weekly and/or monthlies going back much further, for a terabyte of space. That way, in the worst case, you could go back to a backup before the corruption began. Obviously you would want to have plenty of safeguards in place so that that never happened, but just in case, it's good to have an honest to goodness backup too.
This is the big thing I can't figure out what people are not understanding. git does consistency checking for you already, tar|rsync|etc. don't, so it makes sense to take advantage of that.
What we had was an instance of some of the underlying data becoming corrupt on the filesystem (with indications of that starting on Feb 22!). The big mistake was considering the source repositories as consistent and canonical at the remote anongit end, but the data would have been just as corrupt if we had scp'ed the repos from git.kde.org to the anongit mirrors around the world, since we would have bypassed git's internal checking in that way.
Is it safe to rsync a running mysql database at random times, or are you supposed to use mysql-provided tools to perform a backup?