So SSE2 intrinsics should be safe for x86_64 builds?

Yes.

So most Rust builds are inefficient given they don't take advantage of modern instructions?

By default Rust builds binaries that will run on any processor for the given target. If you want it to generate instructions that are processor-specific--thereby reducing the portability of the binary--you must pass an explicit compiler flag. Clang exhibits the same behavior with its -march and -mcpu flags, and I would assume GCC does as well.