Then you better don't use it when you're fatigued or drunk. I nearly locked my SIM card once by not realizing until the third attempt that my phone was asking for the SIM card PIN rather than my lockscreen PIN.
There's also an "admin PIN" that can be used to unlock the key if something goes wrong. It's more complicated than my regular PIN and I didn't memorize it so that should be enough as a defensive measure against drunk me.
Well, unless I'm silly enough to try and bruteforce that PIN as well, after 3 failures I'd be left with an expensive piece of plastic... Fortunately I'm rarely that drunk.
EDIT: Actually as the sibling comment points out you can still reset the token even if you mess up the admin PIN. So at least you won't "brick" your token completely.
Then you better don't use it when you're fatigued or drunk. I nearly locked my SIM card once by not realizing until the third attempt that my phone was asking for the SIM card PIN rather than my lockscreen PIN.