I think it's entirely dependent on the type of headphones. The kind you mention are really slim and sleek; they don't really look out of place. It's when I see people wearing Beats or even bigger over/on ear headphones that attract my attention. I don't think ill of people wearing these headphones, but I do think they look silly because of how unwieldy the headphones seem.
You're correct about the larger cans. Personally, I dislike the weight of those type, and also how unwieldy they are and that they do not produce superior sound to smaller, more intelligently designed models (that usually don't cost more).
