It is perfectly correct to anticipate that military AI is a primary existential threat to humanity.
Either AI has never been invented yet, in which case we will doom our galaxy. This is improbable, given our position and time in the galaxy.
Or AI has been invented, there are other species, and they're largely invisible for some reason. One reason could be that their own AI killed them.
Or we could be under some kind of quarantine for primitive cultures, in which case placing our weapons under AI control would render it trivial for a hostile external actor to exterminate us using our own AI as a proxy.
The fact is that if you don't believe in a higher spiritual reality, then AI is very scary. It is the logical next stage of evolution, and RNA world did not fare well in the transition to DNA world.
> Either AI has never been invented yet, in which case we will doom our galaxy. This is improbable, given our position and time in the galaxy.
Why is that improbable? We have no real idea what is the probability of life starting on a planet, much less evolving to our current level. If that probability is sufficiently low, we being the first species in our galaxy to reach that level would not be improbable at all. Since we don't know what that probability actually is, we can't know that it is improbable.
If the fossil record showed that life were a sickly invalid, perhaps. However, it shows the opposite: life is irrepressible, dynamic, and endlessly adaptable. There is no evidence of Earth languishing sterile for eons before getting lucky with abiogenesis.
Instead we find signs of life on Mars, such as a face next to a vitruvian pyramid. This indicates that life did not start on Earth due to its unique Moon, but that the Moon was parked in a precise orbit by unknown earthly life. Abundant terrestrial indicators corroborate this hypothesis, forming a pattern of covert interference.
The correct threat assessment on AI, therefore, is that it is too vulnerable to hostile non-human manipulation to be trusted with strategic WMDs.
It is an acceleration of the problem inherently posed by WMDs: Achieve international peace or we will all perish together. Why wouldn't the galaxy let such things play out? Nobody wants another warlike species bursting into their neighborhood.
Either AI has never been invented yet, in which case we will doom our galaxy. This is improbable, given our position and time in the galaxy.
Or AI has been invented, there are other species, and they're largely invisible for some reason. One reason could be that their own AI killed them.
Or we could be under some kind of quarantine for primitive cultures, in which case placing our weapons under AI control would render it trivial for a hostile external actor to exterminate us using our own AI as a proxy.
The fact is that if you don't believe in a higher spiritual reality, then AI is very scary. It is the logical next stage of evolution, and RNA world did not fare well in the transition to DNA world.