An attacker with a privileged position on the network allowing them to eavesdrop (but not decrypt) traffic could use a bug like this to identify the device on the network associated with a phone number in Signal. Given nation state level adversaries, that seems like a significant privacy issue to me.