Perhaps you do need to drive 2 miles to answer your phone, but I think the point was that the vast, vast majority of Google's users do not need to do that. (sidenote: isn't that because you have 2-step-auth turned on? how would you like it to work, given your circumstances?)
I don't have two step auth on, but it wanted to check my number anyway, which was fine with me. If they'd set a longer timeout it would have been find with me.
