Colleges don't accept privilege as a currency. Privilege is the contagion you get from going to one. If you're white, and especially male, you incur a lot of privilege-contagion. If you're from a historically underprivileged segment (but not including poor whites), you get negative contagion--you're struggling against great odds and joining the historically privileged, good work!
Where I grew up, there were kids in high school that left high school as soon as they were able to get jobs because their family needed them to help put food on the table.