I think that in reality, the innocent third parties concerned about their health and life on a road such as this are stakeholders. Moreover, there may be more of them so their needs would trump the few who simply wanted to drive fast and reach their destination 2 minutes earlier.

I'm not suggesting that the non drivers should always get their way though. There are roads where cars are allowed to be driven faster than this, and roads where pedestrians are not permitted.