So I have a sample size of 5 people that I know who did this. Still talk to 3 of them. It seems like the lower stress life is temporary. As they grow older they face new challenges that seem to be just as stressful as their work.
My personal belief is that happiness in life comes from being able to deal with adversity rather than away from it - the biggest advantage that it gives you is that you often find there are "have your cake and eat it too" choices you can make where there only seems a binary choice. For example, it may be possible for you to secure a job offer at a start date a year from now, ride out the high paying job, save/invest money, then you go back to a lower paying job while having a nice reserve of money to play with.
My personal belief is that happiness in life comes from being able to deal with adversity rather than away from it - the biggest advantage that it gives you is that you often find there are "have your cake and eat it too" choices you can make where there only seems a binary choice. For example, it may be possible for you to secure a job offer at a start date a year from now, ride out the high paying job, save/invest money, then you go back to a lower paying job while having a nice reserve of money to play with.