I think you're misunderstanding the argument people make about the concept of privilege. Nothing about it is intentional in the standard sense. It's more that people behave to benefit themselves -- which is natural and fine -- and power has slowly accrued over time to white men more than to other types of people. The argument is not that white men are malicious, but that their traits are implicitly favored and promoted because we tend to gravitate toward and feel more comfortable around people who are similar to ourselves. Ergo, white men in power enable other white men to gain more power.