This was pretty much inevitable - there is so much insecure code out there from practically every era of computing. There’s only one answer, and that is giving developers and security experts the resources (that includes LLMs) and time they need to ensure that all this legacy code is being fixed or replaced with better structured code that is less attackable.
It will be a wild ride for a while, and then things will slow down again when the well of not-yet-discovered exploitable bugs starts to run dry.
I’m quite staunchly anti-LLM, but code review is where I think they are most suited. Mind you, only as PART. But their ability to hold large bodies of well-structured code in context is what gives them the edge to find possible bugs and exploits a human has overlooked.
They never should’ve been used for code creation, and in reality they’re being use on both sides of the equation going completely unmonitored or, at best, there’s a burnt out human sitting in between rubber-stamping commits because the entire pitch for this shit is to increase productivity (output) and move at a rate designed to outpace them.
In my experience, AI writes better code (and tests) than most developers and much worse architecture than a senior engineer.
I suspect the architecture is fine, really, but AI is so comfortable juggling 19 balls that it’s architecture never needs simplification and clarity. But as a human reading it, it’s just a mess and humans need much more coherence and encapsulation.
I feel like AI writes good code but then every once in a while it does the absolute dumbest thing in the world. And you have to be really vigilant so you can’t it doing dumb shit.
Then most developers you know are subpar. Frontier models routinely write far worse code than I. And I am only a moderately decent dev.
I have even seen llm code generated with a frontier model that blew straight through array boundaries without a worry in the world.
They can be useful at work, especially for tech I know little about, and as an additional set of eyes to avoid easy mistakes like off by ones, or doing wrong pointer arithmetic, but they are pretty abysmal at code generation.
I thought the same earlier this year. Opus 4.8 mostly produced garbage code in my eyes. Then Opus 5.5 came and now it can routinely produce about the same quality as me but much faster.
True, but what I mean is that it can find ways to do what I need to do even for things that I don’t know a lot about. If the “solution” it finds ends up being a bunch of hacks I change my approach. If it finds supported and documented APIs, I read their documentation and make sure that my mental model of the the things I’ve just learned about matches the code and explanation from the llm, and if they do I rewrite the code in a way that makes sense and iterate from there.
I also don’t copy paste the code and rewrite it inline. I rewrite it in a different window from scratch. The llm is confined to its browser window.
Essentially, I use these things the way I used to use google a few years ago: searching for documentation and examples. That is no longer possible with today’s search engines.
when the well of not-yet-discovered exploitable bugs starts to run dry.
i think that roughly 80% of all code in existence is buggy. humans are just really bad at figuring that out. it’s actually amazing that our software infrastructure stood for this long without crashing down.
This was pretty much inevitable - there is so much insecure code out there from practically every era of computing. There’s only one answer, and that is giving developers and security experts the resources (that includes LLMs) and time they need to ensure that all this legacy code is being fixed or replaced with better structured code that is less attackable.
It will be a wild ride for a while, and then things will slow down again when the well of not-yet-discovered exploitable bugs starts to run dry.
I’m quite staunchly anti-LLM, but code review is where I think they are most suited. Mind you, only as PART. But their ability to hold large bodies of well-structured code in context is what gives them the edge to find possible bugs and exploits a human has overlooked.
They never should’ve been used for code creation, and in reality they’re being use on both sides of the equation going completely unmonitored or, at best, there’s a burnt out human sitting in between rubber-stamping commits because the entire pitch for this shit is to increase productivity (output) and move at a rate designed to outpace them.
In my experience, AI writes better code (and tests) than most developers and much worse architecture than a senior engineer.
I suspect the architecture is fine, really, but AI is so comfortable juggling 19 balls that it’s architecture never needs simplification and clarity. But as a human reading it, it’s just a mess and humans need much more coherence and encapsulation.
Ever seen them do documentation? I wrote better shit in high school
Yeah the documentation is, again, great in the detail and terrible in the overview and store telling.
I feel like AI writes good code but then every once in a while it does the absolute dumbest thing in the world. And you have to be really vigilant so you can’t it doing dumb shit.
For sure. I’d never just accept AI code without serious investigation but it’s a useful tool for good programmers.
Then most developers you know are subpar. Frontier models routinely write far worse code than I. And I am only a moderately decent dev.
I have even seen llm code generated with a frontier model that blew straight through array boundaries without a worry in the world.
They can be useful at work, especially for tech I know little about, and as an additional set of eyes to avoid easy mistakes like off by ones, or doing wrong pointer arithmetic, but they are pretty abysmal at code generation.
I thought the same earlier this year. Opus 4.8 mostly produced garbage code in my eyes. Then Opus 5.5 came and now it can routinely produce about the same quality as me but much faster.
Opus 5.5 with extra effort is what routinely produces worse code than I produce. Still great as a second set of eyes mind you.
Nononono “tech you know little about” means you don’t know enough to see the holes it’s making
True, but what I mean is that it can find ways to do what I need to do even for things that I don’t know a lot about. If the “solution” it finds ends up being a bunch of hacks I change my approach. If it finds supported and documented APIs, I read their documentation and make sure that my mental model of the the things I’ve just learned about matches the code and explanation from the llm, and if they do I rewrite the code in a way that makes sense and iterate from there.
I also don’t copy paste the code and rewrite it inline. I rewrite it in a different window from scratch. The llm is confined to its browser window.
Essentially, I use these things the way I used to use google a few years ago: searching for documentation and examples. That is no longer possible with today’s search engines.
Google is a great comparison actually- you have a slew of information…but you can’t blindly appply it or you’ll wipe win32
i think that roughly 80% of all code in existence is buggy. humans are just really bad at figuring that out. it’s actually amazing that our software infrastructure stood for this long without crashing down.