You beat me to it, as stupid as it is, you don't want to deal with this small minority.

People on the other side who complain about this being an "intrusive, misguided attempt at preventing discrimination" should take the time to talk to them and say hey, this is not about you.