I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?
When did we enter the twilight zone where bug trackers are consistently empty? The limiting factor of bug reduction is remediation, not discovery. Even developer smoke testing usually surfaces bugs at a rate far faster than they can be fixed let alone actual QA.
To be fair, the limiting factor in remediation is usually finding a reproducible test case which a vulnerability is by necessity. But, I would still bet most systems have plenty of bugs in their bug trackers which are accompanied by a reproducible test case which are still bottlenecked on remediation resources.
This is of course orthogonal to the fact that patching systems that are insecure by design into security has so far been a colossal failure.
Bugs are not the same as (real) high severity bugs.
If you find a bug in a web browser, that's no big deal. I've encountered bugs in web browsers all the time.
You figure out how to make a web page that when viewed deletes all the files on the user's hard drive? That's a little different and not something that people discover very often.
Sure, you'll still probably have a long queue of ReDoS bugs, but the only people who think those are security issues are people who enjoy the ego boost if having a cve in their name.
Eh, with browsers you can tell the user to go to hell if they don't like a secure but broken experience. The problem in most software is that you commit to bad ideas and then have to upset people who have higher status than the software dev that would tell them to go to hell.
But, have you any code that has been vetted and verified to see if this approach works? This whole Agentic code quality claim is an assertion, but where is the literal proof?
It’s agents all the way down - until you have liability. At some point, it’s going to be someone’s neck on the line, and saying “the agents know” isn’t going to satisfy customers (or in a worst case, courts).
The fact that KiCad still has a ton of highly upvoted missing features and the fact that FreeCAD still hasn't solved the topological renumbering problem are existence proofs to the contrary.
It probably is introducing more bugs because I think some people dont understand how bugs work.
Very, very rarely is a bug a mistake. As in, something unintentional that you just fix and boom, done.
No no. Most bugs are intentional, and the bug part is some unintended side effects that is a necessary, but unforseen, consequence of the main effect. So, you can't just "fix" the bug without changing behavior, changing your API, changing garauntees, whatever.
And that's how you get the 1 month 1-liner. Writing the one line is easy. But you have to spend a month debating if you should do it, and what will happen if you do.
The pressure to do so will only happen as a consequence of the predicted vulnerability explosion, and not before it. And it will have some cost, as you need dedicated and motivated people to conduct the vulnerability search, applying the fixes, and re-checking until it comes up empty, before each new deployment.
The prediction is: Within the next few months, coding agents will drastically alter both the practice and the economics of exploit development. Frontier model improvement won’t be a slow burn, but rather a step function. Substantial amounts of high-impact vulnerability research (maybe even most of it) will happen simply by pointing an agent at a source tree and typing “find me zero days”.
I feel like the dream of static analysis was always a pipe.
When the payment for vulns drops i'm wondering where the value is for hackers to run these tools anymore? The LLMs don't do the job for you, testing is still a LOT OF WORK.
That might be one outcome, especially for large, expertly-staffed vendors who are already on top of this stuff. My real interest in what happens to the field for vulnerability researchers.
Perhaps a meta evolution, they become experts at writing harnesses and prompts for discovering and patching vulnerabilities in existing code and software. My main interest is, now that we have LLMs, will the software industry move to adopting techniques like formal verification and other perhaps more lax approaches that massively increase the quality of software.
Also, synthetic data and templates to help them discover new vulnerabilities or make agents work on things they're bad at. They differentiate with their prompts or specialist models.
Also, like ForAllSecure's Mayhem, I think they can differentiate on automatic patching that's reliable and secure. Maybe test generation, too, that does full coverage. They become drive by verification and validation specialists who also fix your stuff for you.
Outside of limited specific circumstances, formal verification gives you nothing that tests don't give you, and it makes development slow and iteration a chore. People know about it, and it's not used for lot of reasons.
I've never been on a security-specific team, but it's always seemed to me that triggering a bug is, for the median issue, easier than fixing it, and I mentally extend that to security issues. This holds especially true if the "bug" is a question about "what is the correct behavior?", where the "current behavior of the system" is some emergent / underspecified consequence of how different features have evolved over time.
I know this is your career, so I'm wondering what I'm missing here.
It has generally been the case that (1) finding and (2) reliably exploiting vulnerabilities is much more difficult than patching them. In fact, patching them is often so straightforward that you can kill whole bug subspecies just by sweeping the codebase for the same pattern once you see a bug. You'd do that just sort of as a matter of course, without necessarily even qualifying the bugs you're squashing are exploitable.
As bugs get more complicated, that asymmetry has become less pronounced, but the complexity of the bugs (and their patches) is offset by the increased difficulty of exploiting them, which has become an art all its own.
LLMs sharply tilt that difficulty back to the defender.
Yes and no. Good defence is layered and an attacker needs to find a hole in each layer. Even if it is not layered intentionally a locally exploitable vulnerability gives little if you have no access to a remote system. But some asymmetry does exist.
My sense is that the asymmetry is non-trivial issue here. In particular, a threat actor needs one working path, defenders need to close all of them. In practice, patching velocity is bounded by release cycles, QA issues / regression risk, and a potentially large number of codebases that need to be looked at.
> If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software?
Probably because it will be a felony to do so. Or, the threat of a felony at least.
And this is because it is very embarrassing for companies to have society openly discussing how bad their software security is.
We sacrifice national security for the convenience of companies.
We are not allowed to test the security of systems, because that is the responsibility of companies, since they own the system. Also, companies who own the system and are responsible for its security are not liable when it is found to be insecure and they leak half the nations personal data, again.
Are you seeing how this works yet? Let's not have anything like verifiable and testable security interrupt the gravy train to the top. Nor can we expect systems to be secure all the time, be reasonable.
One might think that since we're all in this together and all our data is getting leaked twice a month, we could work together and all be on the lookout for security vulnerabilities and report them responsibly.
But no, the systems belong to companies, and they are solely responsible. But also (and very importantly) they are not responsible and especially they are not financially liable.
>> If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software?
>Probably because it will be a felony to do so. Or, the threat of a felony at least.
"my software" implies you own it (ie. your SaaS), so CFAA isn't an issue. I don't think he's implying that vigilante hackers should be hacking gmail just because they have a gmail account.
"worth" can have two meanings in this context. $100 from 1917 can be worth exactly $100 today. Or it can be worth what you can buy with it.
Some folks will see a $100 bill from the era and see an old $100 bill. Some folks will imagine what that $100 took to save back then, and what it bought.
FWIW my brain automatically went with "the goods that can be bought with $100" - such as what I could buy in a grocery store today with $100 would be about what I could buy with $3 back then.
I never considered the other reading until this thread. It was obvious to me the author meant "you can buy 97% less stuff today with the same $100".
I think it's used to convey that the buying power has been reduced. If you have a $100 basket of goods (as measured in 1914 dollars), $100 in 1914 allows you to buy 1 basket of goods. Due to the devaluation, today spending $100 would only give you a $3.05 basket of goods (as measured in 1914 dollars).
It's a bit of an odd comparison since you're using two different units for dollars to compare the basket vs purchasing dollar. The clearer way to say it is that today's $100 basket of goods is equivalent to $3.95 basket of goods of 1914.
I didn't realize how much I appreciated writing having a distinct voice until LLMs made everyone sound the same. This strikes me as extremely LLMy:
> SaaS era: ~decade to go upmarket. Cloud era: ~5 years. AI era: <2 years. The gap between 'developers love this' and 'enterprises are asking for SOC 2' has never been shorter.
No judgement if you want to write your articles with LLMs or whatnot, you do you, I've just discovered that their default style grates a bit. It's like when Bootstrap came out, initially it looked amazing but very quickly it became the "default site" look.
I'm at a stage where I don't want to be doing network management on my weekends. I have a Ubiquiti router that's pretty good, and for my router I'd like something like TrueNAS for my NAS, a distribution that completely turns the hardware into an appliance I can configure once and forget about.
Pfsense/opnsense would be one option (based on FreeBSD). For Linux there is OpenWRT, which you can either run as an alternative firmware on quite a few consumer routers/access points, or install on a PC or Pi or similar.
Caveat: I have only used OpenWRT on a high end consumer router (GL.inet MT6000) out of those. That works well, anything else is based on reading about people using those options.
For all of those, once you set it up you don't really need to do much except install updates a couple of times per year, or if you want to forward a new port or such.
My Framework 13 is a bit long in the tooth. I can pay 529 EUR to get a new mainboard and keep the same case/battery/speakers/camera/keyboard/mouse/screen/etc. Or, I can replace the keyboard for 32 EUR.
It's not just repairs, to upgrade a Mac you have to throw away all that perfectly working hardware just to get a new mainboard.
> I can pay 529 EUR to get a new mainboard and keep the same case/battery/speakers/camera/keyboard/mouse/screen/etc.
Or you can spend 50 euros more and get an entire new laptop that is not only much more powerful than your old framework but is almost as repairable: the neo.
At some point your argument begins to work against you, you should just have talked about the keyword repair being cheap. Not how you can get a new motherboard for "only" 530 euros.
> Or you can spend 50 euros more and get an entire new laptop that is not only much more powerful than your old framework but is almost as repairable: the neo.
You forget to mention - less powerful than his old FW 13 with new mainboard/CPU.
I assume he's referring to the AMD AI 340 for 530 euros.[0]
Macbook Neo 31% faster ST speed and a bit slower on the MT.[1]
I wouldn't call the Neo less powerful than his 530 euros upgrade. In fact, I'd much rather have the faster ST speed in this kind of laptop. Most of the apps you're running with this class of laptops will be ST bound anyway.
You can literally get a brand new Macbook Neo using Apple EDU pricing for the price of a slower AMD motherboard upgrade. This is why Framework is an absolutely terrible deal overall. I'm not even convinced that Framework is better for the environment since Apple laptops last extremely long and will very often have second and third hand buyers.
reply