The connection to Amdahl's law is totally on point. If you're just using LLMs as a faster way to get _your_ ideas down, but still want to ensure you validate and understand the output, you won't get the mythical 10x improvement so many seem to claim they're getting. And if you do want that 10x speedup, you have to forego the validation and understanding.
I do agree with you, but don't underestimate the projects where you can actually apply this 10x. For example, I wanted to get some analytics out of my database. What would have been a full weekend project was now done in an hour. So for such things there is a huge speed boost.
But once software becomes bigger and more complex, the LLM starts messing up, and the expert has to come in. That basicaly means your months project cannot be done in a week.
My personal prediction: plugins and systems that support plugins will become important. Because a plugin can be written at 10x speed. The system itself, not so much.
Yes, definitely. I also don't think every project is able to create a plugin platform. Sometimes you just have a lot of interconnected components, where they kind of influence each other.
What I was trying to say is that in future developments, as a developer, one of the extra questions on your mind should be: can we turn this into a platform with separate plugins? Because you know those plugins can be written fast, cheap, and don't require top notch engineering work.
But I think I get what you are saying: what you gain in plugin simplicity, you pay in effort to design the platform to support them.
I guess it will depend from project to project, and so the typical "it depends" applies :).
People have been thinking about that a long time though. For that objective, LLMs don't seem to open up any new capabilities. If that problem could be solved, with really clean abstractions that dramatically reduce context needed to understand one "module" at a time, sure LLMs will then be able to take that an run. But it's a fundamentally hard problem.
NYT has it out for digital advertisers, who directly compete with them. I do sense some schadenfreude here that the tech nerds who work at these places might be in trouble.
"Silicon Valley panjandrums spent the 2010s lecturing American workers in dying industries that they needed to “learn to code."
To copywriters at the NYT, LLMs are far better at stringing together natural language prose than large amounts of valid software. Get ready to supervise LLMs all day if you're not already.
The code is also recognizable as slop to those who know how. Not the tropey "Not X, but Y" kind that's super easy to spot. But tons of repetition, deeply nested code, etc.
A counterpoint is that (maybe) nobody cares if the code is understandable, clean and maintainable. But NYT is explicitly in the business of selling ads surrounded by cheap copy just good enough to attract eyeballs. I suspect getting LLMs to write that is going to be far easier than getting LLMs to maintain large code bases autonomously.
If you explicitly make it go over the code file by file to clean up, fix duplication and refactor, it'll look much better, while no amount of "fix this slop" prompting can fix AI prose.
> no amount of "fix this slop" prompting can fix AI prose
What's the proof for that? What fundamental limitation of these large language models makes them unable to produce natural language? A lot of people see the high likelihood of ever increasing amounts of generated, no-effort content on the web as a real threat. You're saying that's impossible.
>What fundamental limitation of these large language models makes them unable to produce natural language?
LLMs can get indefinitely good at coding problems by training in a reinforcement learning loop on randomly generated coding problems with compiler/unit tests to verify correctness. On the other hand, there's no way to automatically generate a "human thinks this looks like slop" signal; it fundamentally requires human time, severely limiting throughput compared to fully automatable training signals.
In my experience, code validation (unit testing, code review, manual testing, etc.) was more of a bottleneck than producing code for the most part. This means that faster code generation wouldn't produce significant gains in throughput unless the code validation speeds up too. In my workplace, I've seen evidence that the people showing the biggest productivity gains from AI coding are now shipping enormous commits that are barely getting any validation. Given the Zeitgeist, others are for some reason more lenient towards that than they normally would be (or should be).
"As a friend who works in AI told me, AI heightens the contradictions. It is a boon to those with the motivation and background to cultivate knowledge but it spells total destruction for the system of universal education and credentialing. My worry is that we may run out of people with motivation and background to learn, know, and do. In the future, Gen X and millennial knowledge workers will be the human capital equivalent to pre-war steel. Just as particle detectors need steel forged before atmospheric nuclear testing gave all newly forged steel unacceptable background radiation, we will discover that even if your job mostly consists of interacting with LLMs, doing so well will require people who remember what it was like to read and interpret a document or contrast two ideas without asking an LLM to do it for you."
Wide-spread use of LLMs may exacerbate this problem, but it's already been one for a while now. The toxic combination of smartphones and attention-economy short-form video content are already robbing the young of focused effort and the benefits of an attention span longer than a few seconds. It's sad to see things may be about to get far, far worse though.
