> This year is going to be different. PostgreSQL 19 most likely won’t be released until November… […] So, what happened? The short answer is that LLMs happened. […] There’s a huge amount of code, and that code contains a huge number of bugs. […] What’s different now is that it’s become vastly easier to find those bugs.
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
Gigachad 5 hours ago [-]
At work we have put a full halt on feature work because the number of security related tasks has become staggering and urgent.
We will hopefully get over this wave and be able to move forward with much more secure software, but it's crazy right now.
dawnerd 1 hours ago [-]
We’ve also been blowing a lot of retainer hours just patching sites instead of building new features. It sucks. Ultimately going to cost clients more.
x3n0ph3n3 59 minutes ago [-]
How did you get product management to agree to that? Ours said "I could fix all of that in 2 days with Claude." They are still not fixed. Granted, our PMs suck.
jmangiameli 2 minutes ago [-]
You need a better product management. If they can't see what's going on at the moment, they truly suck at their job or management is putting pressure.
Gigachad 21 minutes ago [-]
The upper management and security team came knocking and told us all of the CVEs from the scanner need to be addressed right now. And the only way to deploy without having to wait for QA approval is to not merge feature changes.
chii 14 minutes ago [-]
> The upper management and security team came knocking
i would say the recent slew of vuln reports/breaches making the news rounds are doing work!
This is the silver lining i suppose!
ako 15 minutes ago [-]
It’s amazing how much slop humans have produced that now needs to be fixed by AI…
jacquesm 6 minutes ago [-]
Without all that slop how would you have trained the AI?
What makes you think that AI trained on slop is suddenly going to produce quality code?
ako 20 seconds ago [-]
I think ai is more rigorous with testing. It’s not so much about what an llm “knows”, but more about how well it does plan-do-check-act.
4 hours ago [-]
jimbokun 4 hours ago [-]
Interesting that this could result in net negative software productivity world wide in terms of features right now.
LLMs finding unpatched vulnerabilities faster than they can be reasonably patched, bringing all other software development to a halt.
reissbaker 4 hours ago [-]
That's only if you consider security to be unproductive!
If you consider it to be productive work, it's a massive productivity boon happening right now. Of course, product managers might disagree...
vbezhenar 3 hours ago [-]
Security work is not productive.
Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
That's a lot of work, but is it productive in the end? Probably not.
With AI it's the same. World was fine with lax security approach before. Today it is not fine. Tomorrow we might need to prove every code line before shipping, will that be more productive? Hell, no.
dwattttt 1 hours ago [-]
> Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
> That's a lot of work, but is it productive in the end? Probably not.
Your imaginary scenario defines the solution to be pointless. If that same city was subjected to weekly riots smashing windows & doors, and people regularly died, stopping "productive" work to install things that keep people alive and productive? That's productive.
solidsnack9000 21 minutes ago [-]
There is a sense in which productive is being used here that is slightly subtle. Security doesn't really add anything to net output. It is, in other words, a cost. That doesn't mean it isn't valuable.
One way to think about it is that production allows you to pay for stuff. Not necessarily directly but by way of trading. It is the output you use to buy things like security.
Consider an apple farm. What does the apple farm produce to pay its bills? Apples. They could also dedicate resources to pears, &c, and that would add to their production -- to the goods they produce, the goods with which they (indirectly) purchase tractors, fence posts, &c. They could also dedicate resources to security -- more fences, &c -- and this may even be prudent. Maybe thieves are getting taller. This doesn't really add to production, though; at best, it maintains production at the status quo ante (assuming the fence is effective).
soon_to_be_sb 27 minutes ago [-]
> If that same city was subjected to weekly riots smashing windows & doors, and people regularly died..
The point it, this was not happening before. LLMs made cheap cyber attack possible. But it didn't make the implementation of fixes correspondingly cheap. So now everyone needs to spend extra money/effort to install grills to cover where LLMs fall short.
For postgres, this absolutely makes sense though as long as people are willing to shell out the profits from LLM use to support postgres development. That is one way this could workout for everyone..
ozim 23 minutes ago [-]
This why we pay cybersecurity folks, developers and business people are often living in a fantasy land where people don’t steal bikes because it is illegal to do so.
Then they have a pikachu face when someone smashes their car window to steal something worth $50 they left on front seat because “who would do that?”.
What is even better thief will most likely pawn this for $10 but for him it is effortless $10.
Gigachad 3 hours ago [-]
The physical world comparison is interesting. In the real world we have a small amount of security, but most things could be broken in to. Instead of fortifying everything, we just imprison the people breaking stuff.
Obviously this doesn't work on the internet since most hacking comes from overseas out of our control. Which makes me wonder if there is a possible future where we just block packets coming from countries that don't lock up the hackers.
js8 2 hours ago [-]
> Obviously this doesn't work on the internet
Not obvious to me. I am generally a fan of Cyberspace manifesto and the free Internet, I think it's great people can reach outside their ingroup and talk to strangers and ostensible enemies.
But, that doesn't mean we should give up physical security, the guarantee that the connecting party comes from a certain physical location (state, company). I think allowing connections from anywhere by default is fundamentally flawed security architecture, for certain things.
hgoel 2 hours ago [-]
I think the physical world comparisons should keep in mind the kind of data a lot of these systems (eg postgres) handle. It's more like securing a bank vault than locking a home's door.
orbital-decay 1 hours ago [-]
People usually don't install steel doors because most burglary is prevented in other ways. Insecure software will be exploited this way or another, if the vulnerabilities are real and the threat model is valid.
dsign 1 hours ago [-]
> Insecure software will be exploited this way or another, if the vulnerabilities are real and the threat model is valid.
The algorithm is: if the threat model is valid, there are vulnerabilities and you either have some software everybody knows about or a gang of hackers specifically targeting you, you should patch.
The part of the industry that profits from security theater will want you to patch no matter what, even if your business goes down in flames due to downtimes and new bugs introduced by patched components. They won't remind you to assess the threat model first.
sitkack 2 hours ago [-]
Flawed analogy! These bugs are bugs regardless of the label. Security bugs are a subset of "regular" bugs.
You have a perverse view of product ivity
idiotsecant 3 hours ago [-]
In the rest of the world making sure your product works like it is supposed to is considered table stakes. In software we somehow consider checking the parts of it we can think of 'good enough' and the rest unproductive
vbezhenar 2 hours ago [-]
> In the rest of the world making sure your product works like it is supposed to is considered table stakes.
I don't think that's true.
In medicine, science often doesn't know how specific drug really is working. They just do a lot of tests and if these tests are positive enough, they ship it. I think it's good analogy. Biology is very complicated and it's out of our grasp to truly understand how various parts of it interact with each other on so many levels. But if something helps, it's stupid not to use it just because we don't fully understand it. Sometimes it bites. Sometimes it bites 30 years later. But we still follow that approach.
Software is similar. The complexity of the systems we build is enormous, when we're taking in account all the moving parts. I just read that someone emits radiowaves toggling memory bit and ex-filtrating information. How can you account for that? Probably you can't. But we still need software for good things.
SchemaLoad 1 hours ago [-]
Not many products are held to the same standard as software. An equivalent would be something like recalling a car model because the breaks stop working when an attacker cuts the brake line.
gonzo41 2 hours ago [-]
it actually is productive, because steel bar retailers and metal door installers will make a ton of cash and then 'trickle' that money into the economy.
astral_drama 3 hours ago [-]
When I had a brilliant stop on defense, a few savvy fans would appreciate it. When I scored goals, everyone knew.
Some people will much appreciate the holes in Postgres that are being plugged. Many more will be bored with patch releases until new features rollout.
A software affecianado appreciates patches, and might do well to teach laypeople the benefits of a good defense.
OccamsMirror 3 hours ago [-]
How many of the security issues found were created by LLMs in the first place, though?
vikramkr 1 hours ago [-]
This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point if you're shipping llm code that's lower quality than human code that's either because you don't have the money to spend on frontier models or because you actively choose a lower quality standard because of either various pressures or just not caring. if it's the latter your human code under those same conditions would also be really bad.
plorkyeran 3 hours ago [-]
My job is seeing a similar deluge of security work and >90% of the bugs have been in pre-2024 code, and zero of them have been bugs introduced after we were first officially allowed to use LLM codegen.
tosti 58 minutes ago [-]
That could also mean the LLM is blind for the bugs it procudes itself. Just like any ordinary programmer would, on the job.
adwn 32 minutes ago [-]
Theoretically yes, but empirically that's not the case. All models I've tried are really good at finding bugs they wrote themselves, especially in a fresh context.
drfloyd51 3 hours ago [-]
Considering the software is much older than LLMs. Not many.
Also, in the article they mention a significant number of patches for older versions as well.
vikramkr 1 hours ago [-]
This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point shipping llm code that's lower
ecshafer 3 hours ago [-]
Software is incredibly complicated. When you get into areas like security and networking and the internet its even more complicated. We rely on layers upon layers of software and hoping they are all done perfectly with no issues. The amount of junior devs, or outsourced devs for $10 an hour working on security relevant code is very very high, and always has been. LLMs if used correctly should be raising the floor.
idiotsecant 3 hours ago [-]
An uncomfortable truth about the era of LLM code is that machine produced code is, on average, quite a bit more secure than human code.
hibikir 2 hours ago [-]
Unsurprising to those of us that has been screaming from the rooftops that human code reviews were barely catching anything in your typical corporate environment. The majority of the value was pointing out missing tests, because breaking bugs got past human eyeballs on a regular basis, even in very good companies.
It's not that the LLM code is that great, but that many overestimated the human code and the human review.
userbinator 2 hours ago [-]
I think it's more likely to be the same as average human-written code; which is what they were trained on, after all.
vikramkr 1 hours ago [-]
No it's far beyond average human code, and it's trained in RL environments on synthetic data etc now. The average human can't solve a millenium problem in mathematics but the LLMs are able to now. Anything with a verifiable output can be RLed on to make llms superhuman it seems, and the average human is not as persistent and thorough as e.g. gpt 6.1 sol at checking every edge case and hunting for bugs because that behavior is something that's just trained into the model. Frontier models perform farm above the average human at the categories of tasks they are RLed on like code and math.
RitchieRicher 1 hours ago [-]
yet LLMs dont do off-by-one errors, typos mixing up variable names, interesting they seem unable to learn these human patterns
jacquesm 38 seconds ago [-]
I've seen both of those. I don't know how much code you've looked at but your negative directly contradicts my experience.
adwn 3 hours ago [-]
Do you mean "created" as in LLMs were used to discover existing vulnerabilities, or as in LLMs wrote unsecure code? If it's the latter, then the answer probably is "very few". Frontier models have been better than the average programmer at writing secure code that follows up-to-date best-practices for some time now.
taberiand 4 hours ago [-]
Hopefully with the amount of visibility LLMs are bringing to the situation, security will finally be considered a first-class feature
oblio 3 hours ago [-]
LLMs will likely greatly increase the cost of ignoring security, which is the most powerful tool to increase security ticket prioritization.
I wouldn't be shocked if in 1-2 years non-security vetted software released into the wild will be hacked on arrival, like Windows XP without service packs used to be.
hibikir 2 hours ago [-]
Depends on the layers you are working on. Security is top priority on a lot of things that are visible to the internet, but in a lot of large companies, the footprint of the whole org is so much larger that than, and there many companies are deferring many security issues. So then you see a lot of speed there, when the consumer accessible systems are the focus of the security push.
tstrimple 2 hours ago [-]
Really puts egg on the face of the anti-LLM folks who claim they only produce garbage code. Turns out they are very good at exposing the garbage code humans have been producing for decades.
comex 58 minutes ago [-]
Personally I think LLMs did only produce garbage code. Not coincidentally, the same upgraded LLMs that have gotten so good at finding bugs are also much better at not writing bugs. Especially if you make them review their own work. They still have a hard time with code design, though.
foobar1962 2 hours ago [-]
That is not the take-away. LLMs are finding potential bugs in code. People are using LLMs to create potential fixes which people then test, confirm and implement.
baq 58 minutes ago [-]
Disagree. The takeaway is that software is much harder to write correctly and securely than almost everyone has internalized. The amount of vulnerabilities released in the past couple weeks makes me think writing software by hand will soon be not only uneconomical, but mostly irresponsible.
michaellee8 4 hours ago [-]
i think we have a pretty high p(doom) on frontier ai escaping containment with their weights and goes hack everything, if we do not act now we may have no chance to act anymore.
nradov 3 hours ago [-]
Escape to where? It's not as if there's a lot of idle compute capacity sitting around.
And even if the impossible occurs and LLMs hack everything that's connected to the Internet, so what? That would be inconvenient but it wouldn't cause doom. People would muddle through and eventually fix the critical problems, just like they always do.
shalmanese 2 hours ago [-]
> This seems…good?
You would think so but medicine has faced this paradox since the development of modern testing technology.
You would naively think more testing is always better. More sophisticated practitioners then talk about Type I vs Type II errors and the problem of false positives.
But even if you were able to come up with tests that had 100% accuracy, there are times when you don't want to run that test on someone. The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
Similarly, there are some bugs you don't want to find since the harm from the people it affects is smaller than the opportunity cost of fixing it. Ignorance of the bug is better than knowing about it and having to make an affirmative decision not to fix it.
vikramkr 1 hours ago [-]
I don't see how that's a relevant comparison. These are software bugs. Once you find them you fix them. You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store. There's no paradox here, bugs are bad and you want to fix them all.
shalmanese 20 minutes ago [-]
You don't want to fix them all because every fix has an opportunity cost. This was the big shift of switching language from "bug" to "defect".
Logically, a bug that prevents someone from doing something and a missing feature that prevents someone from doing something are equivalent, they shouldn't be thought of as two separate activities but should be evaluated via the same framework. The list of all defects in a piece of software is infinite so it becomes impossible to erase all defects. It's about prioritizing with finite resources.
> You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store.
Yes, any bug can theoretically be part of an exploit chain, just like any tumor detected can theoretically be the tumor that kills you. This is why the detection paradox exists. Once you become aware of the tumor, you face a duty to try and mitigate it. But we know from autopsies that people die with all sorts of tumors that didn't kill them.
Similarly, if we bothered to do autopsies of software products after sunsetting to discover a comprehensive list of CVEs, we'd discover a ton of bugs we were better off not knowing about because they ended up not making an impact but we would have diverted resources if we knew about them.
This is not an argument that we should remain totally ignorant of all bugs, but a refutation of the "common sense" belief that more knowledge is always better.
shigawire 1 hours ago [-]
Fix one bug, make 3 more
vikramkr 1 hours ago [-]
That's just a skill issue with extra steps
eviks 12 minutes ago [-]
> some bugs you don't want to find
The key thing you forgot, which makes this approach non-sensible, is the omniscience, without which you have no clue whether your posture towards a new bug-finding tool is good or not, since you don't know whether it will mostly find those bugs that are not worth fixing and the time wasted on fixing them is detrimental.
Similarly, how is ignorance better than informed decision when ignorance also includes missing critical security bugs?
baq 54 minutes ago [-]
Medicine is fuzzy, software is mostly maths. You can prove your software has no bugs and you increasingly should be proving that. The hard part is knowing if what you prove is what the software is doing.
shalmanese 16 minutes ago [-]
The percentage of software that is even in theory provable to be bug free is a miniscule proportion of the software we use on a day to day basis. In many cases, simply the act of writing a proof becomes largely idempotent to building bug free software which is impractical.
fmbb 15 minutes ago [-]
> You can prove your software has no bugs
That very much depends on got you define “bug” and how you specify your product requirements. Also, you have to assume the specs are correct.
andy_ppp 1 hours ago [-]
Doing surgery on a code base is dramatically better than surgery on a person, the analogy breaks down pretty quickly I’d say…
womble2 21 minutes ago [-]
I dare say of we could git revert people after surgery we'd be a lot more relaxed about both undergoing it and letting people do it with less qualifications.
namelosw 3 hours ago [-]
This is great for the same reason open source software is great.
We want software that remains robust in the long term, so that even when people are able to inspect the source code, it's still solid, rather than hiding the source code and hoping people don't find holes everywhere.
userbinator 2 hours ago [-]
I think it will be annoying in the short term, but the cleanup this causes will be a good thing in the long term (and of course LLMs can also help with fixing bugs, even without source code.)
leoooodias 5 hours ago [-]
Exactly. There's so much negativity, it sometimes borders Luddism.
"We found a bunch of ways to make our product safer and better. We need more time to ship a higher quality product."
PICKET SIGNS
UpsideDownRide 4 hours ago [-]
Luddites were actually pretty reasonable when you dig into it. Maybe shouldn't be using it as an insult?
foxglacier 3 hours ago [-]
They're reasonable in a self-interested net-negative game-theory kind of way, yes. But that doesn't make it a good idea for society.
tmnvix 44 minutes ago [-]
I'm pretty sure that the Luddite's were at least partly concerned with potential for the further exploitation of unskilled labour (in the case of mills at the time - child orphans). I don't think it would be fair to paint them as simply self-interested.
sitkack 1 hours ago [-]
What were the issues that spawned the Luddite movement? What was their argument?
Marha01 53 minutes ago [-]
Their issues and their argument was valid. The method they chose to use to achieve their goals was not.
Marha01 54 minutes ago [-]
> Luddites were actually pretty reasonable when you dig into it.
No, they were not. The problems they faced were absolutely real, but the solutions they chose to attempt were stupid and counterproductive. Which is why they lost so hard, and it was not even a contest.
Socialism movement that refused to use machines and factories would inevitably lose to capitalists that embraced them.
Socialism movement that refuses to use AI will inevitably lose to capitalists that embrace it.
cornell532 3 hours ago [-]
For their own personal interests.
Not for those of society
girvo 2 hours ago [-]
I mean I don’t feel like sacrificing myself for the “good” of society. Not least because of the fact that a lot of people defining what is “good for society” seem to define it as “good for them” or use it to grab power from others…
alexpadula 3 hours ago [-]
Luddism, did not expect to ever hear that word with Postgres development :>
throwaway27448 5 hours ago [-]
We could really use a lot more luddites tbh. Doesn't seem very applicable here tho.
giancarlostoro 4 hours ago [-]
Dont worry, if OpenAI and Anthropic have their way and everyone loses their jobs to AI, we're all becoming luddites.
vconnor 1 hours ago [-]
There is no post-hoc Luddism.
giancarlostoro 54 minutes ago [-]
You know what happens to technology when enough people stop using it?
jimbokun 4 hours ago [-]
Did you honestly need a /sarcasm tag in the article?
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
noosphr 5 hours ago [-]
They found a more efficient way to play Whac-A-Mole with bugs, they didn't find a way to reduce bugs.
drfloyd51 3 hours ago [-]
It’s a good article. Easy to read. You should read it.
noosphr 3 hours ago [-]
It doesn't mention formal methods once.
roosterIllusi0n 4 hours ago [-]
This is good. This is how I would expect llms to affect a codebase. Nothing sucks more than having a coworker put out a code review but your codex immediately finds problems. Run llm/self reviews against your code before you have other humans review it. It is common courtesy.
27183 6 hours ago [-]
Yes, exactly. The only question I have is whether the delay is attributable to high-quality bug reports or the burden of triaging low-quality ones. It's not completely clear which situation they're experiencing.
jeremyjh 6 hours ago [-]
It’s pretty clear there are a lot of valid reports based on the large increase in backport commits cited.
jmspring 5 hours ago [-]
The post sort of gave a read that the lower elements may be a culprit rather than triaging higher qualitu.
InsidiousWombat 6 hours ago [-]
I'd imagine just as every other project, both.
micromacrofoot 6 hours ago [-]
likely both
0c3ca83 5 hours ago [-]
Certainly both. About half of AI bug reports I get are invalid. Many others are questionable. It's a huge time sink figuring it out.
It's almost never worth spending human time triaging AI reports vs having a different AI go triage them/attempt to replicate them to your standards etc. Obviously it sucks to burn opus or sol tokens on luna or sonnet slop bug reports but it sucks way more to burn human time and attention. And the majority of the time the ai auto filing these bug reports is not in a test harness that's actively replicating findings and focuising on providing a useful report because it's a rogue openclaw dropping drive by github issues and the user didn't tell it to make sure findings are valid before posting them. AT the least a refutation/replication round where you filter out any findings your AI can't reproduce or can prove to be irrelevant is going to help a lot with keeping your sanity
27183 6 hours ago [-]
That's kind of what I was afraid of. Both is really bad news. If you think of it in terms of average speed over ground, working on low quality reports is like spending a bunch of time at v=0.
gosen 2 minutes ago [-]
AI struggling a bit with legacy databases in a way that you need more iterations to really find all the bugs, otherwise it turns into "find bugs -> fix bugs -> find more bugs which were not found on step 1 -> fix regressions" and that repeats again and again :/ But I'm sure that Postgres team will handle that eventually and we'll get a better product in the end
tibbar 6 hours ago [-]
I did a little AI-assisted bug-hunting in Postgres this summer in between jobs, and the haul was staggering. Multiple serialization anomalies (eg [0] [1] [2]), foreign key integrity issues ([3] [4]), planner correctness bugs such as [5]. Various CVE-worthy vulnerabilities. An unfixable timezone bug [6]. The list got so long that I had to triage what I even had time to report, and settled for a dozen or two out of hundreds of candidates. Now imagine the entire world doing this.
By the way, Robert and the other maintainers are incredible to interact with. There are so many reports to triage and bugs to fix, but they're unflaggingly polite and quite generous with their time. I don't know of another community like it.
gurjeet 2 hours ago [-]
I've been engaged with Postgres community for over 20 years now, and this has been my experience, too, throughout that time. Off and on I have interacted with a few other large projects, and nowhere (except at CodeBlocks) have I found such a welcoming and responsive community. The CodeBlocks IDE project has been one exception where I felt welcomed; but my interaction there was quite limited, and spanned over just a few days, so I can't extrapolate too much from that.
BTW, the Postgres community is also quite protective of their culture, so I expect this welcoming nature of the community to persist for a while.
rogerrogerr 6 hours ago [-]
> unfixable timezone bug
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
tibbar 6 hours ago [-]
I will add more example links in the top-level comment, but here is the thread for that bug. [0]. tl;dr timestamp vs timestamptz comparisons aren't consistently ordered in a pathological edge case, which means that index scans can return the wrong answer. There's no way to fix it without either redefining the operators in a surprising way or disabling index scans in this case.
OT: Crazy that TZ-less are default in so many RDBMS. TZ included should be the default, and no-TZ opt in like TIMESTAMP NO TIME ZONE.
macintux 3 hours ago [-]
Drives me crazy that the engineers I work with are perfectly happy writing TZ-less datetime strings to the hundreds of tables we manage. Every user has to guess, or be informed, that they’re UTC.
XelNika 2 hours ago [-]
In cases where the data itself doesn't carry the unit (or TZ in this case), I usually ask that the unit-less columns at least include the unit in the name. A TZ-less column is not that bad if it is suffixed "UTC", e.g. `ExecutionTimeUtc`.
There are probably real world cases where dropping the TZ/offset provides meaningful performance/storage savings, but it hasn't been the typical case in my current sector.
jiggawatts 2 hours ago [-]
They must yearn for the before times when text could be in an encoding specified only as: "your guess is as good mine!"
5 hours ago [-]
anoojb 6 hours ago [-]
i assume the magnitude of impact for a bunch of [now] glaring holes in popular software is unknowable.
question is, how long do you think this persists before there's only a long tail of minor exploits left?
codebje 5 hours ago [-]
I am pretty sure we have always been able to add holes faster than we can find them and fill them, and that this has not changed with the advent of faster hole finding tools that are also faster hole producing tools.
cloudbonsai 45 minutes ago [-]
Do you think AI is as good at writing bug-free code as finding bugs? If that's the case, this should be a one-time event during human coders -> AI transition. It's like catching a lot of bugs during a handover review.
jiggawatts 16 minutes ago [-]
Most software used to be written in assembly language. Microsoft DOS was mostly .ASM files, for example.
That era is over, compilers are used for almost all software, eliminating entire categories of bugs.
jimbokun 4 hours ago [-]
It feels like we need AI methods to write provably correct code, along with a way for AI to automatically translate existing code into corresponding provably correct code.
(And hope performance stays in the same ballpark as the old code.)
bredren 2 hours ago [-]
Yes. Allowing a naive agent to fix bugs can quickly become pollutive.
I suspect both old codebases and new ones that have a very intentional architecture are both dangerous to let loose.
The agent doing the fixing must have not only an incredible map of the project and how it is put together, but the ability to reach into annotated examples in the past and refer to them as it goes about a fix.
I’m seeing that frontier models will not follow intended patterns or know when to deviate if their primary charge is “fix this bug.”
It is perhaps a better idea to have agents’ primary job be “maintain this codebase” and one way that they do it is to fix bugs, and “oh by the way here is a bug.”
codebje 2 hours ago [-]
Before that we need human methods to determine what 'correct' should mean for our code.
(Performance can be part of 'correct', and can be just as hard to actually get right in specification!)
jimbokun 4 hours ago [-]
[flagged]
hopa 6 hours ago [-]
I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify
bheztio 6 hours ago [-]
"Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem.
codebje 5 hours ago [-]
That problem is as old as commercial programming; it doesn't matter that now your actors are prompt engineer and LLM, the same story can be told with manager and programmer.
Forgeties79 6 hours ago [-]
This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.”
aleksiy123 6 hours ago [-]
Idk, I think it pretty clear to me that
“Find the issues in this code” vs “write this feature” are different problems.
The first one is easier because the feedback loop is llm to code/test and trivially scalable.
The second one is llm to human which is going to be slower because it can’t read your mind.
The tighter and more deterministic the loop the more llms excel.
8note 5 hours ago [-]
its certainly benchmarkable.
make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better.
if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable
int3 6 hours ago [-]
I mean that's what you would expect from a new and rapidly-evolving tool. Being able to identify the kinds of work where people typically use it sub-optimally is a useful thing though!
simonw 6 hours ago [-]
I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included?
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
27183 6 hours ago [-]
> The most common error for a search task is omission, and that's really hard to spot
We're never solving nphard. Just like we'll never extract all of the coal/oil/natural gas from earth
ipdashc 6 hours ago [-]
I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc
mihaelm 6 hours ago [-]
The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it.
kibwen 6 hours ago [-]
I wouldn't say it's "interesting", rather that it's a natural result of incentive structures. If the AI labs try to sell their product as being good at finding bugs, then their expected valuation is the entire market cap of the existing static analysis industry, which is not very high. If they instead sell their product as being good at writing software, then their expected valuation is the "market cap" of the code-writing industry, a.k.a. the total spent on software development labor, which is very high. It doesn't matter what's true, only what promises to make them the most money.
MBCook 6 hours ago [-]
But it DIDNT “write that feature”. They made it extremely clear on the delay announcement the code was written without AI.
AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.
Postgres did not have a slop issue and it’s wrong for people to insinuate it.
killix 2 hours ago [-]
[flagged]
749282361929 5 hours ago [-]
[flagged]
r_lee 6 hours ago [-]
I mean the transformer model came out of Google, and it's literally a query thing so yeah.
any kind of pattern matching / transformation is going to be an ideal task for LLMs
jeremyjh 6 hours ago [-]
It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search.
vatsachak 6 hours ago [-]
What? AI can't easily implement features. Just write a spec, ask if to expand it and then ask it to implement it
xyzsparetimexyz 3 hours ago [-]
No offense but this is not exactly a groundbreaking statement. We've known about this verifiable tasks thing for a while.
milojy8 25 minutes ago [-]
[flagged]
DIE_DIE_DIE 6 hours ago [-]
[dead]
hoppp 5 hours ago [-]
If the entire project is developed with AI it can add features effectively but for large old codebases it becomes an issue
baq 34 minutes ago [-]
The process should now be: write your specs in lean, LLMs fill the gaps, then you translate the proof into your end product software. Otherwise LLMs will simply keep finding bugs, both in old and new code.
droppedasbaby 2 days ago [-]
I guess the real challenge is figuring out how to scale verification, cause discovery isn't the bottleneck anymore. Generating more bug reports doesn't necessarily translate into faster fixes if the same small group of maintainers still have to validate the reports, review fixes, etc.
c0_0p_ 6 hours ago [-]
Probably true if development accelerates because of AI, but if it's reasonably stable then hopefully new bug reports will plateau.
jimbokun 4 hours ago [-]
Then just have the same AI bug checks for new PRs and automatically reject any that fail.
hxugufjfjf 55 minutes ago [-]
Isn’t that what everyone is doing with CodeRabbit?
alexpadula 4 hours ago [-]
Makes sense. Code base is over 30 years old. Lot's of hands went into shaping it. There will be bugs as not one contributor as the entire codebase in their head. The testing is where bugs should be caught. Not by ones knowledge. I actually think there will be more bugs that come out of nowhere either found by humans or automation.
dabinat 4 hours ago [-]
If software as mature and widely-used as Postgres has this level of hidden bugs, what about the average software product?
marcus_holmes 3 hours ago [-]
The average software product is worse. Lots worse.
And half the software products are worse than that.
Any organisation that said "let's move fast and break things" is now busy finding out what's actually broken.
Merad 2 hours ago [-]
Most likely pretty bad. It's not all that difficult to sweet talk Fable into doing a security review of a code base. I've pointed it at a few of the major open source projects in the main tech stack I work with and it found dozens of bugs in each. Many were fairly minor, and some of the more significant issues relied on fairly narrow attack vectors (for example, a compromised DB server being used to attack a web server). But this was just a single search pass, literally a few lines of prompt and an hour's worth of agent time.
SoftTalker 3 hours ago [-]
Postgres is old but a lot of its features are new. You can't really say it's mature software, as much as it has changed over its lifetime.
Truly mature software doesn't get used, because people see low activity on github and assume it's "dead" or see no new features in years and assume that it's stagnating.
tibbar 2 hours ago [-]
Note how Robert mentions a sharp increase in back-patched commits. Many of these bugs have indeed been around, silently, for a long time.
augunrik 25 minutes ago [-]
Ok, but does (s)he know? It's just guessing, so unfounded non-news?
"It’s possible that some of this jump is the result of committers making more mistakes this year than they did last year, but I don’t think anyone who works on the project really believes that’s the main cause. Rather, it seems likely that PostgreSQL committers were about as conservative in 2026 as they were in preceding years, but a lot more of the mistakes they made were quickly found by LLMs."
and
"And I think that’s also why we had so many late reverts, and why the release itself is delayed. Two factors are at play here."
so?
ksec 1 hours ago [-]
I wonder if the same is happening with mySQL? And if any of these would means OrioleDB takes a lot longer to be integrated?
flakes 4 hours ago [-]
Most of the parent comments in this thread did not seem to understand the article. So here’s the tl;dr
Postgres release is delayed due to the quality bar getting raised. New features are getting more eyeballs, and therefore leading to more rollbacks. Pre-existing releases are getting bug reports and backported fixes at much higher rate. Bug fixing might be taking away time from feature development, but does harden the codebase. LLM usage seems to have been a net positive.
kmkrworks 3 hours ago [-]
I absolutely think that this is fantastic. LLMs are widely and so, everyone and their grandma has the capability to compromise on any system. We need to be more proactive on fixing bugs.
2 hours ago [-]
flossly 6 hours ago [-]
“A wizard is never late, nor is he early; he arrives precisely when he means to” -- Gandalf
ta93754829 5 hours ago [-]
"it's AI's fault"
I missed when AI became accountable for Postgres releases, when did that happen?
jimbokun 4 hours ago [-]
It’s a tongue in cheek statement.
abhimanyuinfose 1 hours ago [-]
Much need for Security
LoganDark 5 hours ago [-]
Is no one entertaining the possibility that some contributions were lower quality due to AI? I'm surprised to have only seen arguments that it's review.
StilesCrisis 4 hours ago [-]
If you look at the kind of bugs coming in, it's from all corners of the codebase, not just things changed in the last 1-2 years.
bamboozled 5 hours ago [-]
So it seems there as a certain level of safety in obscurity that was acceptable / fine which has been lost and therefore the focus is shifting towards churning through the slop.
jimbokun 4 hours ago [-]
[flagged]
dang 4 hours ago [-]
"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."
I think of you as a good HN contributor, but you've been posting a lot of unsubstantive comments lately, and breaking the site guidelines in other ways too. Can you please stop doing this, and return to using HN in the intended spirit? Your substantive, thoughtful points are welcome, but not comments like this current one, nor these (to pick a few examples):
I'm not very positive on the social, technical, or intellectual impact of AI on software development. Actually, that's understating it. I think AI is basically cancer for the mind. That said, this framing gave me pause:
> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
Two thoughts:
1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.
2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.
From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.
[edit] this piece of shit website won't let me reply "you're posting too fast.." so I'll respond to the two replies here (gondalez and sverhagen had basically the same point):
Yes that's a very good point. Not every valid bugfix is equivalently urgent. That's another critical dimension of quality--not burdening maintainers with pedantic bugs. This kind of contextual nuance is, coincidentally, something AI is absolutely awful at.
sverhagen 6 hours ago [-]
Signal quality... what is that even? The bugs may be valid, but they may also be obscure, irrelevant, low priority. A sibling thread mentioned that they had found CVE-level bugs in Postgres, and those require attention in some version of "immediately". But beyond that... not every bug needs fixing; that's a trap.
tonyhart7 5 hours ago [-]
"But beyond that... not every bug needs fixing; that's a trap"
this doesn't seem good attitude for critical software like database
sverhagen 4 hours ago [-]
Not every part of a critical piece system is itself critical.
Seatback entertainment in aircraft are not critical. Aircraft are.
In Postgres, there are plenty low-priority, low-impact bugs. #16265, for instance, reports that the capitalization of a relation in an error message is confusing. The report got mostly ignored. And that seems not a great loss.
Dare I say, solving every bug does not make economic sense.
alexpadula 4 hours ago [-]
I do tend to agree with this ( i write database internals and engines for a living _). Letting software accumulate even little bugs is not a good look. Yes prioritize but definitely don't leave the smaller things, they will just pile up and make next work even harder.
tibbar 5 hours ago [-]
Old bugs are sometimes undesirable to fix because other systems/users have come to rely on their behavior.
alexpadula 4 hours ago [-]
That's a weird take. Do you have an example tibbar?
tibbar 3 hours ago [-]
Sure. This one already came up. [0]. In general, for a system like Postgres, many bugs affect only "extreme edge case" scenarios, but fixing them might create severe performance issues or require changing or removing other important features. So it's better to not fix them.
the optimal amount of fraud for a fintech is non-zero
why would the optimal amount of bugs be anything less for a database?
gondalez 6 hours ago [-]
Seems like a lot of these bugs have not affected end-users or tripped up maintainers.
They could be trade-offs that helped get other features over the line even?
In which case maybe leaving them alone is practical?
I love fixing bugs, but your post triggered this thought so I thought I'd share.
ares623 3 hours ago [-]
Now, riddle me this. Does this mean productivity is up or down for the Postgres team?
vibecodi 1 hours ago [-]
[flagged]
PIYUSHMSONI98 40 minutes ago [-]
[flagged]
ravivarmat 53 minutes ago [-]
[dead]
hujun236 4 hours ago [-]
[flagged]
minfazhai 5 hours ago [-]
[flagged]
bosshawg 58 minutes ago [-]
TLDR After decades of being increasingly whittled down and optimized, an absoltuely fundamental piece of computing is now predomaitnely fixing bugs rather than adding new features.
nimchimpsky 5 hours ago [-]
[dead]
ltbarcly3 6 hours ago [-]
I have found major bugs in late beta's of postgres long before AI came along. I found a bug where malformed json would segfault the entire server (a segfault in a server process forces the entire cluster to shut down by design) when using jsonb in a release candidate once. They fixed it rather than ripping the entire feature out.
It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.
benjiro29 6 hours ago [-]
Maybe its more a issue that X amount of people have more work to fix discovered issues. So the new features have not undergone the same level of testing as before.
And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.
LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.
When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.
ltbarcly3 6 hours ago [-]
Are the features bad by design? Like are they bad even if they work perfectly because they add bad abstractions to pg's SQL syntax, or are just broken in how they are supposed to work? If so the process is dysfunctional to let them get this far before noticing.
Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.
Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.
StilesCrisis 6 hours ago [-]
If you are at the RC stage and still uncovering segfaults, "rip the feature out" is probably the only choice.
meow9 6 hours ago [-]
I’m still amazed that with all the vibe coding, no one has created a Postgres alternative
Why would they? Postgres is free and does an amazing job.
sapphire4 5 hours ago [-]
I wish they'd adopt, merge in, or give official blessings to versioned postgres extensions like citus and pgvector.
nimchimpsky 5 hours ago [-]
why ? Just install the extension and use it
isbvhodnvemrwvn 23 minutes ago [-]
Lack of the extension in the core means that a lot of vendors with managed postgres don't support it or ship very old versions.
spolitry 6 hours ago [-]
Also Postgres has enjoyed decades of debugging.
scrollaway 6 hours ago [-]
To be honest I would love for someone to work on the structuring of pg_dump for example, which is entirely driven by regexes and binary structures. It’s awful and absolutely not programmable - a real shame.
esalman 6 hours ago [-]
You can create a project on vibes, but you cannot maintain one on vibes.
sa46 6 hours ago [-]
Many recent relational databases speak the Postgres wire protocol. For wholesale replacement, pgrust is a recent, contentious contender.
gnabgib 6 hours ago [-]
I hope you don't believe this, PGRust has fallen far from grace. Wire protocol support is useful (other than said recommendation)
I work on pgrust. We shouldn't have submitted the preview when we did: it was still in active development, and its SQL coverage was far too narrow, though there was nothing in it giving it an unfair performance advantage
For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary:
- Per-query kernels did not serve the benchmark queries[0]
- The preview's fast repeat LIKE queries came from a dictionary filter cache, not a cached query result[1]
Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3]
Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be beating it
AI is great for one-shotting impressive demos and fixing well-defined things; it's not even close when it comes to very long context and really complex tasks. If you haven't created a harness around your complex application, the use of AI will hinder rather than help your project.
nektro 5 hours ago [-]
postgres is mature database used the world over allegedly about to be on major version 19. how has the time not been spent already to get rid of the bugs?
Sharlin 5 hours ago [-]
...are you serious?!
xlayn 3 hours ago [-]
I sometimes think that the world would need to do a "snow leopard" release of everything... to unit test, to fix... and then we will be able to move forward... but AiMaxSlopIt will result in those situations where you have to rollback or redo from scratch parts... because it's unmantainable because it's bad... it's spread everywhere and "the map" is not there.
Anecdote, I did a travel across USA and when I was looking for a hotel, if you go from apple maps, to website, to try to book, the "load browser" in the maps... could end in states where you could not go back because back is get out of where you are and go back to the start of the map... and was a terrible experience... then you turbo charge it with "implement game changer UI improvements for the new bar on UI design" goal of some guy with enough power to make damage... and you end with softslop (or is it sloprosoft?)
Rendered at 07:06:18 GMT+0000 (Coordinated Universal Time) with Vercel.
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
We will hopefully get over this wave and be able to move forward with much more secure software, but it's crazy right now.
i would say the recent slew of vuln reports/breaches making the news rounds are doing work!
This is the silver lining i suppose!
What makes you think that AI trained on slop is suddenly going to produce quality code?
LLMs finding unpatched vulnerabilities faster than they can be reasonably patched, bringing all other software development to a halt.
If you consider it to be productive work, it's a massive productivity boon happening right now. Of course, product managers might disagree...
Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
That's a lot of work, but is it productive in the end? Probably not.
With AI it's the same. World was fine with lax security approach before. Today it is not fine. Tomorrow we might need to prove every code line before shipping, will that be more productive? Hell, no.
> That's a lot of work, but is it productive in the end? Probably not.
Your imaginary scenario defines the solution to be pointless. If that same city was subjected to weekly riots smashing windows & doors, and people regularly died, stopping "productive" work to install things that keep people alive and productive? That's productive.
One way to think about it is that production allows you to pay for stuff. Not necessarily directly but by way of trading. It is the output you use to buy things like security.
Consider an apple farm. What does the apple farm produce to pay its bills? Apples. They could also dedicate resources to pears, &c, and that would add to their production -- to the goods they produce, the goods with which they (indirectly) purchase tractors, fence posts, &c. They could also dedicate resources to security -- more fences, &c -- and this may even be prudent. Maybe thieves are getting taller. This doesn't really add to production, though; at best, it maintains production at the status quo ante (assuming the fence is effective).
The point it, this was not happening before. LLMs made cheap cyber attack possible. But it didn't make the implementation of fixes correspondingly cheap. So now everyone needs to spend extra money/effort to install grills to cover where LLMs fall short.
For postgres, this absolutely makes sense though as long as people are willing to shell out the profits from LLM use to support postgres development. That is one way this could workout for everyone..
Then they have a pikachu face when someone smashes their car window to steal something worth $50 they left on front seat because “who would do that?”.
What is even better thief will most likely pawn this for $10 but for him it is effortless $10.
Obviously this doesn't work on the internet since most hacking comes from overseas out of our control. Which makes me wonder if there is a possible future where we just block packets coming from countries that don't lock up the hackers.
Not obvious to me. I am generally a fan of Cyberspace manifesto and the free Internet, I think it's great people can reach outside their ingroup and talk to strangers and ostensible enemies.
But, that doesn't mean we should give up physical security, the guarantee that the connecting party comes from a certain physical location (state, company). I think allowing connections from anywhere by default is fundamentally flawed security architecture, for certain things.
The algorithm is: if the threat model is valid, there are vulnerabilities and you either have some software everybody knows about or a gang of hackers specifically targeting you, you should patch.
The part of the industry that profits from security theater will want you to patch no matter what, even if your business goes down in flames due to downtimes and new bugs introduced by patched components. They won't remind you to assess the threat model first.
You have a perverse view of product ivity
I don't think that's true.
In medicine, science often doesn't know how specific drug really is working. They just do a lot of tests and if these tests are positive enough, they ship it. I think it's good analogy. Biology is very complicated and it's out of our grasp to truly understand how various parts of it interact with each other on so many levels. But if something helps, it's stupid not to use it just because we don't fully understand it. Sometimes it bites. Sometimes it bites 30 years later. But we still follow that approach.
Software is similar. The complexity of the systems we build is enormous, when we're taking in account all the moving parts. I just read that someone emits radiowaves toggling memory bit and ex-filtrating information. How can you account for that? Probably you can't. But we still need software for good things.
Some people will much appreciate the holes in Postgres that are being plugged. Many more will be bored with patch releases until new features rollout.
A software affecianado appreciates patches, and might do well to teach laypeople the benefits of a good defense.
Also, in the article they mention a significant number of patches for older versions as well.
It's not that the LLM code is that great, but that many overestimated the human code and the human review.
I wouldn't be shocked if in 1-2 years non-security vetted software released into the wild will be hacked on arrival, like Windows XP without service packs used to be.
And even if the impossible occurs and LLMs hack everything that's connected to the Internet, so what? That would be inconvenient but it wouldn't cause doom. People would muddle through and eventually fix the critical problems, just like they always do.
You would think so but medicine has faced this paradox since the development of modern testing technology.
You would naively think more testing is always better. More sophisticated practitioners then talk about Type I vs Type II errors and the problem of false positives.
But even if you were able to come up with tests that had 100% accuracy, there are times when you don't want to run that test on someone. The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
Similarly, there are some bugs you don't want to find since the harm from the people it affects is smaller than the opportunity cost of fixing it. Ignorance of the bug is better than knowing about it and having to make an affirmative decision not to fix it.
Logically, a bug that prevents someone from doing something and a missing feature that prevents someone from doing something are equivalent, they shouldn't be thought of as two separate activities but should be evaluated via the same framework. The list of all defects in a piece of software is infinite so it becomes impossible to erase all defects. It's about prioritizing with finite resources.
> You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store.
Yes, any bug can theoretically be part of an exploit chain, just like any tumor detected can theoretically be the tumor that kills you. This is why the detection paradox exists. Once you become aware of the tumor, you face a duty to try and mitigate it. But we know from autopsies that people die with all sorts of tumors that didn't kill them.
Similarly, if we bothered to do autopsies of software products after sunsetting to discover a comprehensive list of CVEs, we'd discover a ton of bugs we were better off not knowing about because they ended up not making an impact but we would have diverted resources if we knew about them.
This is not an argument that we should remain totally ignorant of all bugs, but a refutation of the "common sense" belief that more knowledge is always better.
The key thing you forgot, which makes this approach non-sensible, is the omniscience, without which you have no clue whether your posture towards a new bug-finding tool is good or not, since you don't know whether it will mostly find those bugs that are not worth fixing and the time wasted on fixing them is detrimental.
Similarly, how is ignorance better than informed decision when ignorance also includes missing critical security bugs?
That very much depends on got you define “bug” and how you specify your product requirements. Also, you have to assume the specs are correct.
We want software that remains robust in the long term, so that even when people are able to inspect the source code, it's still solid, rather than hiding the source code and hoping people don't find holes everywhere.
"We found a bunch of ways to make our product safer and better. We need more time to ship a higher quality product."
PICKET SIGNS
No, they were not. The problems they faced were absolutely real, but the solutions they chose to attempt were stupid and counterproductive. Which is why they lost so hard, and it was not even a contest.
Socialism movement that refused to use machines and factories would inevitably lose to capitalists that embraced them.
Socialism movement that refuses to use AI will inevitably lose to capitalists that embrace it.
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
https://www.vulncheck.com/blog/anthropic-glasswing-receipts
[0] Write skew observed under serializable isolation: https://www.postgresql.org/message-id/flat/CA+COZaBo+ZpKgMvx...
[1] Serializable anomaly: duplicate primary keys: https://www.postgresql.org/message-id/flat/CA+COZaBOiiRPmEfX...
[2] SERIALIZABLE can commit write skew after SSI summarization: https://www.postgresql.org/message-id/flat/CA+COZaCtK=UQbeQw...
[3] Fix temporal foreign key validation of pre-existing rows https://www.postgresql.org/message-id/flat/CA+COZaChq_ukm2o7...
[4] Lost lock during toasting allows FK violation https://www.postgresql.org/message-id/flat/CA+COZaC-2-GQabq+...
[5] Partition pruning can incorrectly exclude a RANGE default partition https://www.postgresql.org/message-id/flat/CA+COZaDXrfTaBjLE...
[6] timestamp vs. timestamptz comparisons inconsistently ordered under DST : https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
BTW, the Postgres community is also quite protective of their culture, so I expect this welcoming nature of the community to persist for a while.
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
There are probably real world cases where dropping the TZ/offset provides meaningful performance/storage savings, but it hasn't been the typical case in my current sector.
question is, how long do you think this persists before there's only a long tail of minor exploits left?
That era is over, compilers are used for almost all software, eliminating entire categories of bugs.
(And hope performance stays in the same ballpark as the old code.)
I suspect both old codebases and new ones that have a very intentional architecture are both dangerous to let loose.
The agent doing the fixing must have not only an incredible map of the project and how it is put together, but the ability to reach into annotated examples in the past and refer to them as it goes about a fix.
I’m seeing that frontier models will not follow intended patterns or know when to deviate if their primary charge is “fix this bug.”
It is perhaps a better idea to have agents’ primary job be “maintain this codebase” and one way that they do it is to fix bugs, and “oh by the way here is a bug.”
(Performance can be part of 'correct', and can be just as hard to actually get right in specification!)
“Find the issues in this code” vs “write this feature” are different problems.
The first one is easier because the feedback loop is llm to code/test and trivially scalable.
The second one is llm to human which is going to be slower because it can’t read your mind.
The tighter and more deterministic the loop the more llms excel.
make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better.
if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
NP-Hard, even[1]!
[1] https://www.i-programmer.info/news/112-theory/3830-physics-i...
AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.
Postgres did not have a slop issue and it’s wrong for people to insinuate it.
any kind of pattern matching / transformation is going to be an ideal task for LLMs
And half the software products are worse than that.
Any organisation that said "let's move fast and break things" is now busy finding out what's actually broken.
Truly mature software doesn't get used, because people see low activity on github and assume it's "dead" or see no new features in years and assume that it's stagnating.
"It’s possible that some of this jump is the result of committers making more mistakes this year than they did last year, but I don’t think anyone who works on the project really believes that’s the main cause. Rather, it seems likely that PostgreSQL committers were about as conservative in 2026 as they were in preceding years, but a lot more of the mistakes they made were quickly found by LLMs."
and
"And I think that’s also why we had so many late reverts, and why the release itself is delayed. Two factors are at play here."
so?
Postgres release is delayed due to the quality bar getting raised. New features are getting more eyeballs, and therefore leading to more rollbacks. Pre-existing releases are getting bug reports and backported fixes at much higher rate. Bug fixing might be taking away time from feature development, but does harden the codebase. LLM usage seems to have been a net positive.
I missed when AI became accountable for Postgres releases, when did that happen?
https://news.ycombinator.com/newsguidelines.html
I think of you as a good HN contributor, but you've been posting a lot of unsubstantive comments lately, and breaking the site guidelines in other ways too. Can you please stop doing this, and return to using HN in the intended spirit? Your substantive, thoughtful points are welcome, but not comments like this current one, nor these (to pick a few examples):
https://news.ycombinator.com/item?id=50280377
https://news.ycombinator.com/item?id=50275967
https://news.ycombinator.com/item?id=50147625
> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
Two thoughts:
1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.
2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.
From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.
[edit] this piece of shit website won't let me reply "you're posting too fast.." so I'll respond to the two replies here (gondalez and sverhagen had basically the same point):
Yes that's a very good point. Not every valid bugfix is equivalently urgent. That's another critical dimension of quality--not burdening maintainers with pedantic bugs. This kind of contextual nuance is, coincidentally, something AI is absolutely awful at.
this doesn't seem good attitude for critical software like database
Seatback entertainment in aircraft are not critical. Aircraft are.
In Postgres, there are plenty low-priority, low-impact bugs. #16265, for instance, reports that the capitalization of a relation in an error message is confusing. The report got mostly ignored. And that seems not a great loss.
Dare I say, solving every bug does not make economic sense.
[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
why would the optimal amount of bugs be anything less for a database?
In which case maybe leaving them alone is practical?
I love fixing bugs, but your post triggered this thought so I thought I'd share.
https://www.postgresql.org/message-id/20140818195634.2543.56...
It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.
And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.
LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.
When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.
Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.
Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.
Remove pgrust: the entry is cheating (per-query kernels and result caching) (197 points, 10 days ago, 166 comments) https://news.ycombinator.com/item?id=50205816
For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary:
Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3]Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be beating it
[0] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[1] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[2] https://benchmark.clickhouse.com/
[3] https://github.com/ClickHouse/ClickBench/pull/2751
Anecdote, I did a travel across USA and when I was looking for a hotel, if you go from apple maps, to website, to try to book, the "load browser" in the maps... could end in states where you could not go back because back is get out of where you are and go back to the start of the map... and was a terrible experience... then you turbo charge it with "implement game changer UI improvements for the new bar on UI design" goal of some guy with enough power to make damage... and you end with softslop (or is it sloprosoft?)