Skip to content
Veristria
See the evidence
All posts

Output outgrew review

For most of software's history, code was written and read at roughly the same rate. That proportion has broken, and the failures it produces are structural rather than careless.

Lars O. Horpestad7 min read
aiengineering

The proportion that broke

For most of software's history, code was written and read at roughly the same rate, for a reason so obvious it never needed stating: the same people did both. Somebody wrote a function, somebody read it in review, and the two activities took comparable amounts of time because they were the same kind of activity performed by the same kind of mind. Every practice we built on top of that assumption — pull requests, pairing, review checklists, the entire cultural apparatus of "somebody else has looked at this" — quietly depended on it holding.

It no longer holds. A small team using AI tooling now produces more working code in a week than it can meaningfully review in a month. That sentence is doing more work than it appears to, so it is worth being precise about which part is new. Teams have always shipped code that nobody read carefully; deadlines have always existed. What is new is that the volume has increased by a factor large enough that careful review is no longer a thing you fall short of occasionally. It is a thing you cannot do at all, arithmetically, and everyone involved knows it while continuing to click approve.

And the code works. This is the part that makes the situation genuinely difficult rather than merely uncomfortable. If AI-generated code were obviously bad, the problem would be self-correcting — it would fail in staging, it would break tests, it would produce the kind of visible mess that creates organisational pressure to slow down. Instead it compiles, it passes, it does what was asked, and it ships. The feedback loop that would normally tell a team it has exceeded its capacity has been removed, because the signal that loop depended on was failure, and failure is exactly what is no longer happening.

What we are left with is a system that appears to be working better than ever while accumulating a category of defect that nothing in the pipeline is positioned to notice. Not because anyone was careless. Because the mechanism that used to catch it was a human being reading the diff with enough time and enough context to ask what it implied, and that mechanism has been quietly starved.

The code works, which is what hides the gap

Consider what actually appears in a review when this goes wrong.

A feature needs data from a table. The query returns nothing, because access rules are doing their job. The assistant, asked to make it work, uses the elevated credential instead, and the feature immediately works. To be reachable from the browser that credential needs a build-time prefix, so a new environment variable is created with a name that looks exactly like every other environment variable in the project.

Now read the diff. It contains an identifier — a variable name, referenced in one place. There is no credential in it. There is no high-entropy string for a scanner to match on. There is nothing that a careful reviewer, reading at the speed a modern pull request demands, would stop on, because in the source nothing is wrong. The secret does not exist yet. It comes into being forty minutes later on a build runner, when a bundler performs a textual substitution and writes the literal value into a file that a content delivery network then serves to every visitor.

This is why "review more carefully" is not the answer, and why I am suspicious of anyone who offers it as one. The failure is not located in the diff. It is located in the relationship between the diff and the artifact, and no amount of attention paid to the first will reveal the second. You could review that pull request for an hour and find nothing, because there is nothing there to find.

The same pattern recurs across surfaces that have nothing else in common. A generated schema arrives complete — tables, relations, indexes, sensible names — and the access policies that should accompany it are absent, because nothing in the request mentioned who was allowed to read the data and a schema is complete on its own terms without them. A refund is issued correctly, and the transfer that moved the seller's share is not reversed, because reversing it is a separate parameter that defaults to off and no error is raised when it is omitted. In each case the artifact under review is correct. In each case the deployed result is not.

AI writes the code. Something still has to check what shipped.

There is a temptation, when describing this, to slide into a tone that blames the reader for using the tools. I want to be careful about that, because it is both unkind and wrong. The assistant reaching for the elevated credential is not incompetent — it is taking the shortest correct-looking path to the outcome it was asked for, which is what it is supposed to do. The engineer approving the diff is not lazy — they are reading a diff in which nothing is wrong. These are structural failures. They happen to good teams, they happen to people who know better, and they will keep happening as long as the only place anybody looks is the source.

What has to change, and what does not

Start with what does not. Nothing about this argues for shipping less, and I would distrust the version of it that did. The productivity gain is real, the tools are good, and the correct response to a process that has been outgrown is to fix the process rather than to throttle the thing that outgrew it. Teams that respond to this by adding review ceremony will lose the advantage and keep the exposure, which is the worst available outcome.

What changes is where the checking happens. If the failure exists in the deployed artifact and not in the source, then some part of your assurance has to move after the deploy, because that is the first moment the thing being assessed exists. This is not a novel idea in other disciplines — you inspect the manufactured part, not only the drawing — but it remains unusual in software, where we have a deep cultural preference for catching things early and a corresponding assumption that anything caught late represents a failure of the earlier stage. Here, late is simply where the evidence is.

The second change is what counts as an answer. If you are going to check the deployed result, the output of that check has to be something you can act on without trusting whoever produced it. A score is not that. A severity label is not that. What you need is the file, the query, the ledger line — the specific thing, in a form you can open yourself and confirm. Anything less reintroduces exactly the problem you were trying to solve, which is that somebody has told you a system behaves a certain way and you have no independent means of establishing whether that is true.

The third change is cadence, and it is the one most often skipped. A check performed once is a true statement about one day. The next deploy produces a new artifact, a new dependency arrives, a new table is created, a rotation misses one environment. Posture decays without anybody making a mistake, which means the useful question is never "did we audit this" but "what changed since the last time we looked".

None of that is dramatic. It is the ordinary engineering response to a proportion that has shifted: notice which assumption stopped holding, and put something in the place it used to occupy. Review is not coming back to parity — the volume will not go down. What can happen is that the deployed result stops being the one artifact nobody ever reads.

Why we exist

Veristria builds verification infrastructure for teams shipping software faster than they can review it. See the three products.