Heavy metal toxicity is basically the same as radiation exposure. It's extremely hard to purge the body of heavy metals after you're exposed, and it's one of the reasons plutonium is as dangerous as it is - because it gets incorporated into your bones. It would be surprisingly safe if it just passed right through you (hence the reason it can be safely handled).

You don't just "get over" heavy metal poisoning. It has long term effects exactly like radiation, and is more insidious because you can't just wave a geiger counter around to find it.