We kind of actually trade with ants - in some forests there are structures for them and we build wooden structures to protect them, in exchange for them keeping the ecosystem safe from rotting animals and kind of cleaning / rotating biomass
I'd be more than happy to trade with ants if they were willing to. They could clean floors and furniture, get rid of plant parasites, check the status of wiring and plumbing. I'd pay them in sugar, that would be a great deal for both parties.
They do the first two (clean and eat bugs) and will happily take sugar in the form of bread an cake crumbs you leave out on surfaces. We're already quite symbiotic
> ... superintelligence would likely not trade with humans because we'd simply be too weak and stupid. That is, if we've got something of value to the AI, it could easily take it from us using manipulation, coercion or force ...
trade is manipulation
if you give ants sugar water in order for them to behave that's useful to you is it trade or manipulation?
> Trade - as we'd recognize it - implies consent,
Manufacturing this consent is a large part of the business of trade.
We don't have explicit trade with ants, they don't explicitly trade with us, yet the ants are doing just fine. One of the most prevalent and prolific species of life on the planet Earth, Ants will arguably be here long after the stupid humans and their super intelligent computers are long gone.
This is pure speculation and not really very well thought through IMO.
We don't kill all ants either.
Even humans work with the principle of collateral damage with each other so if the idea is that AI would suddenly be without mistakes then that's already missing the point of what intelligence is. But just like humans any AI that is "super intelligent" will also need to have morals and ethics (or it can't figure out what IT want and doesn't want) and there is nothing that indicates they shouldn't be extensions of ours as we will be interfacing with them long before any superintelligence comes along, in fact it just as likely that it's our morals and ethics and culture that gets passed on and that biological humans don't get obliterated by AI but simply die out over time.
There is nothing bad about that, our genes survived.
While I will not argue the basic premise, I believe such comparisons always miss a crucial point: Humans, in contrast to almost all other animals, are able to reflect and learn. We can ask ourselves "What if?" better than anything else.
While an AI that's magnitudes smarter than humankind is likely to arise at some point - smarter in a way that will be hard / impossible for us to comprehend - dealing with a species that is able to evolve itself in a self-guided way will be very different than dealing with a colony of instinct-driven ants.
I think a great deal of the result lies in whether we figure out how to design in a conscience. It is not clear that raw intelligence would ever prioritize cooperation with what it saw as a potential threat. Existential calculus the likes of which we see in the Three Body universe seems more likely than some unforeseen magnanimity.
What I don't see being asked is "so what?" Why would trade be important in this relationship? I'd much more expect them to see us and treat us as we treat our aged parents -- not very useful, but we still care for and protect them.
As for the question of should we stop AI research before this happens -- only if the world would have been a better place if insects could have stopped mammals from developing.
