Presumably, when these people bought their homes, they felt they'd be able to live with a relatively continuous sense of community, not feel forced to sell as a billionaire's compound encroached on them.
There's an assumption that these homeowners are getting bought out above market, but what's the market rate for a multi-million dollar home next to the perpetual construction and noise of a billionaire's fife, on a street where an increasing number of homes are being bought out and lay vacant? And why would the property team not negotiate any sale somewhat aggressively?