If you're straight and the other person is the same sex and also straight, you're OK being alone together. If you're gay and with a person of the opposite sex, you're OK being alone together. Anything else is risky.
Anecdotally, I've found myself in situations which would easily qualify as sexual harassment more than once. Each situation would have been "OK" by your definitions, on the surface.
Trying to judge risk in terms of gender or apparent orientation seems a bit silly to me.
