Deleting the keys afterward isn't really a key (heh) part of the attack, just a nice way to cover your tracks afterward. Enough accounts have multiple expired or forgotten keys to make the mere presence of multiple keys and potentially some unknown keys not always an absolute indicator of compromise, too.
It's likely that your ssh key only authenticates your account anyway. If that were the case, you could change your key to a different account, make a change, and then change it back and no one would be the wiser (though any logs demonstrating an identical key across users would identify it, but unlikely they exist and in a form worth pursuing). Security is hard.
Such an attack is unlikely, yes, but still possible.