Say public key x receives 1 BTC in block A. I'm guessing it's encoded as ripemd160(sha256(o)) -> 1 BTC -> ripemd160(sha256((x)) where o is some other public key with sufficient funds. I create a new key pair with public key y, such that ripemd160(sha256((x)) = ripemd160(sha256((y)). From now on let's call this address hash h.

In block B, I make a transaction h -> 1 BTC -> s. Where s is a securely generated public key that I own. I then sign this transaction with my forged public key, which hashes to h.

How does this not give me x's money?

If you can create a pair of (private key, public key with collision), then yes bitcoin will be broken.