As far as the CPU is concerned, there isn't a big difference if you access a block of memory via pointers or via an array. Converting the hash result to a memory address is not the bottleneck. The main bottlenecks are the hash function itself and the logic for dealing with collusions
Nasty stuff that you can do in a language like C :)