Eww that's weird, ok in your case you are out of luck. That's pretty strange though, I have a ton of USB-A devices here and they are all in line with the way I outlined. So that's a bit of a surprise but it may well be an exception.

As for the ones that are sideways: the 'bottom' of the case is the bottom of the motherboard, so if you think of it that way it might still work for all but the top side ones.