If an alien colonization fleet is in orbit around Earth, that's a fair assumption. They didn't come all those lightyears just to have tea and biscuits. They're not likely to just move on just because there are some buffalo already living there. We wouldn't.
It still depends - if you travel lightyears with a sizeable fleet just to capture a specific planet for living space alone, something is seriously wrong with your race anyway (unless you have some yet unknown transportation technology) as you could likely build uncountable space habitats for all that mass and energy spent & right at the heart of your civilization, not somewhere in the sticks with no friendly infra where Earth is.
You need a deeper purpose our you would not be there.
