Well it won't work if everyone goes to the same place. Most places can tolerate (and maybe even benefit from) a small influx of wealthy outsiders.

I think it's somewhat common for people to move back to where they came from if they get tired of big-city life.