You're not really wrong, and what you're talking about is not really related to language choice.
What its related to is the narrative you shared in your second paragraph, which I think you were writing mostly for color, perhaps seeming to you as the only or necessary story of how software gets made.
"And then", "And then", "And then"... "One feature at a time, you just keep stacking more and more on."
There's no foresight. There's no planning. There's no architecture. There's no readiness. There's no preparation. Tomorrow's needs are always beyond the horizon and today's are to be approached strictly on their own terms.
If you've taught software development to anyone, you're used to seeing this behavior. Your student doesn't know what they don't know (because you haven't taught them yet), and they're overwhelmed by it, so they just scramble together whatever they can to make it past the next most immediate hurdle.
Much (not all) of the industry has basically enshrined that as the way for whole teams and organizations to work now, valorizing it as an expression of agility, nimbleness, and efficiency, and humility.
But it inevitably results in code and modules and programs and services and systems where nothing fully snaps together and operational efficiency is lost at every interface, and implementations for new features/challenges need to get awkwardly jammed in as new code rather than elegantly fleshed out as prefigured opportunities.
So you're right that most modern projects eventually just become swamped by these inefficiencies, with no ripe ways to resolve them. But it's not because of Rails vs Go or something, it's because pretty much everyone now aspires to build cathedrals without committing to any plans for them ahead of time.
What they gain for that is some comfort against FOMO, because they'll never face yesterday's plan conflicting with tomorrow's opportunity (since there is no plan). But they lose a lot too, as you call out very well here.
Dynamic scripting languages are orders of magnitude worse than static languages at this problem, though, for several reasons. Their culture tends to encourage these sorts of operations. They're slower than static languages in the first place, meaning every additional layer is that much more expensive on the clock. Their feature sets afford this style of programming, whereas static languages afford much lower cost features, if not outright zero-cost features.
If you're going to be concerned about the sort of things you mention, then as an engineer, you need to turn those concerns into significant negative value placed on the option of dynamic scripting languages.
Which is not to say that static languages are immune... indeed, I marvel at times at the effort and ingenuity put into bringing these issues at great cost into the static languages. Log4Shell, for instance, fundamentally stems from Java, with great effort, importing a very dynamic-scripting-language style reflection feature from the dynamic world, and then getting bitten hard by it. That's not a performance issue in this case, just a vivid example of that sort of thing. You can with enough effort layer enough frameworks and indirection in any language to make anything slow.
"There's no foresight. There's no planning." Couple that with "as an expression of agility," and it really rings true to me. I've worked in enough shops where the contractual obligations preclude any ability to slow down and put together a plan. A culture where you're forced to go from an angry phone call from the suits to something running in production in mere hours is a culture that finds building bookcases out of mashed potatoes acceptable.
The best environment I've ever worked in was, ironically enough, fully invested in Scrum, but it wasn't what's typical in the industry. Notably, we had no bug tracker[0], and for the most part, everyone was expected to work on one thing together[1]. We also spent an entire quarter out of the year doing nothing but planning, roleplaying, and actually working in the business problem domain. Once we got the plan together, the expectation was to proceed with it, with the steps executed in the order we agreed to, until we had to re-plan[2].
With the rituals built in for measuring and re-assessing whether our plan was the right one through, e.g., sprint retrospectives, we were generally able to work tomorrow's opportunity into the plan that we had. With the understanding that successfully delivering everything we'd promised at the end of the sprint was a coin toss, if we were succeeding a lot, it gave us the budget to blow a sprint or two on chasing FOMO and documenting what we learned.
0: How did we address bugs without a bug tracker? We had a support team that could pull our andon cord for us whenever they couldn't come up with a satisfactory workaround (based on how agonizing it was for everyone involved) to behavior that was causing someone a problem. Their workarounds got added to the product documentation, and we got a product backlog item, usually put at the top of the backlog so it'd be addressed in the next sprint, to make sure that the workaround was, e.g., tested enough such that it wouldn't break in subsequent revisions of the software. Bad enough bugs killed the sprint and sent us to re-plan. We tracked the product backlog with Excel.
1: Think pairing but scaled up. It's kinda cheesy at first, but with everyone working together like this, you really do get a lot done in a day, and mentoring comes for free.
2: As it went: Re-planning is re-work, and re-work is waste.
This has been a real challenge in the early startup setting where everything is built to a prototypical standard with the belief that you’re basically buying time until runway is no longer a problem.
And whatever truth there is in that, it’s all out of balance and lacks pragmatism. Sometimes the pragmatic choice isn’t to keep slapping shit together, it’s to step back and see what the big picture is. Even if you don’t have PMF yet, you need to design for something on the off chance it succeeds or you need to pivot.
And it’s not even a pure tech issue, it can be as simple as thinking how a team of engineers could contribute, how you could onboard someone new and have them up and running quickly, what happens if you fire someone who kept everything in their head…
What its related to is the narrative you shared in your second paragraph, which I think you were writing mostly for color, perhaps seeming to you as the only or necessary story of how software gets made.
"And then", "And then", "And then"... "One feature at a time, you just keep stacking more and more on."
There's no foresight. There's no planning. There's no architecture. There's no readiness. There's no preparation. Tomorrow's needs are always beyond the horizon and today's are to be approached strictly on their own terms.
If you've taught software development to anyone, you're used to seeing this behavior. Your student doesn't know what they don't know (because you haven't taught them yet), and they're overwhelmed by it, so they just scramble together whatever they can to make it past the next most immediate hurdle.
Much (not all) of the industry has basically enshrined that as the way for whole teams and organizations to work now, valorizing it as an expression of agility, nimbleness, and efficiency, and humility.
But it inevitably results in code and modules and programs and services and systems where nothing fully snaps together and operational efficiency is lost at every interface, and implementations for new features/challenges need to get awkwardly jammed in as new code rather than elegantly fleshed out as prefigured opportunities.
So you're right that most modern projects eventually just become swamped by these inefficiencies, with no ripe ways to resolve them. But it's not because of Rails vs Go or something, it's because pretty much everyone now aspires to build cathedrals without committing to any plans for them ahead of time.
What they gain for that is some comfort against FOMO, because they'll never face yesterday's plan conflicting with tomorrow's opportunity (since there is no plan). But they lose a lot too, as you call out very well here.