PR review when everyone has LLMs

5 September 2026

The game has changed, and what I thought was a good way to do PR reviews no longer works well when everyone is equipped with modern LLMs.

Over the years PRs have become a ceremony serving multiple functions: catching bugs by getting two or more sets of eyes on it, helping new hires understand the wider codebase and house style, being aware of what your teammates are working on, among other things. Many of these outputs are… just not as important any more. Thanks to LLMs, there are faster and more efficient ways to catch simple bugs, understand the codebase, and track your teammates.

I don’t think we should get rid of PRs. They still have a purpose but it has changed, and therefore the way we conduct PRs should also change.

In this post I want to share the main changes in how I operate on LLM-heavy repos. Obviously everybody’s still figuring things out. I expect in a couple of years much will have changed again.

Reading the code

Until recently my basic position has been that everyone needs to read the code. There are AI enthusiasts who say that’s unnecessary. I have a lot of respect for antirez, who has made this case, so I take the possibility seriously. Despite these voices and despite increasingly powerful models, I don’t think we’re quite there yet. The code is the ground truth and as of 2026 I believe we still need some connection to the code to be responsible and effective product developers.

I know full well that if you ask Claude Code to build a JSON API endpoint that runs a SQL query and outputs the results as JSON, it’s just going to do it right. It’s not going to mess that up. You have it add automated tests, you have it add documentation, you know it’s going to be good.

But I’m not reviewing that code. And now I’ve got that feeling of guilt: if I haven’t reviewed the code, is it really responsible for me to use this in production?

That was Simon Willison on 5 May. Nowadays I’d go further—for a given well-specced coding task, 5.6 Sol will write it about as neatly as I would, with much greater awareness of edge cases. It won’t get tired and skimp on either manual validation or unit tests. It will do it in a few minutes rather than take me all day. For major segments of my work output I literally trust its code more than my own. It’s a bit rough to admit, as someone who’s been paid to write code for around 18 years, but here we are.

I like to think of it like Waymo—there is a point at which the computer will statistically produce fewer defects than a human doing the same task, and I feel we’re reaching that. I haven’t even tried Astra yet.

Today, for a team with LLMs, I think the pragmatic approach is somewhere in the middle: read some of the code[1]. For a given PR to merge I would like to see:

  1. The author has asked their own LLM to do a comprehensive self-review before taking it out of draft[2].
  2. The author has read and understood the key parts of the code. Does the diff fix the bug? What functions implement the new feature and where are they called?
  3. The author has skimmed the auxiliary code and tests. Has it edited some unrelated module? Have 300 lines of code sprung up to cover some case that isn’t all that important?
  4. The reviewer understands the problem, the situation before and after, and has had an active conversation with their LLM about the suitability of this branch.

In particular, I don’t necessarily think reading the code is a good use of the reviewer’s time and energy. Certainly they should ask their own LLM to do an adversarial review and find code-level flaws, but that’s the less important part.

Often as a reviewer I will have conversations like this with my chatbot[3]:

<Human> Review this PR. Explain the problem from a product or user-facing perspective, give me a refresher on relevant context, explain what was changed and why that solves the problem. Highlight any bugs or code quality issues that I should bring up.

<GPT> … [This is the problem, this is how it was solved, the code looks pretty good] …

<Human> Okay so what if the user did X? Wouldn’t it kind of suck in that scenario

<GPT> [traces it through] Yes you’re right, this solution would handle that poorly because […]

<Human> What if we injected this data at this earlier point? Would that fix it?

<GPT> […] Yes

Now I have a high-value comment to put on the PR.

Obviously this is just a pseudo-conversation but you get the idea. The highest value I can bring as a reviewer is to catch architectural and product-level missteps. Today, this awareness generally sits outside the context of the LLMs that are writing the code so that’s the human’s job[4].

I would go further and say that the architectural/product review was always the most valuable part of the review. Previously, reading the code was usually a prerequisite to be able to comment well on the architecture. If you were lucky there was a good PR description but it’s rare that it’s complete and clear enough to cover all relevant discussions.

When an LLM ingests a PR diff and the surrounding context, that session has become a live and interactive PR description. You can ask for extra detail about the bits you care about. You can check what assumptions have been made. You can learn exactly what it is that you want to know about the PR so that you can bring your expertise to bear as to whether it’s a good idea to merge it. It’s almost unfair to expect the author of the PR to produce some static text upfront that serves the same need for all possible reviewers. Better to just put a clear summary on the PR and then the reviewers can have their LLM pull up whatever context they need.

This is a significant change to what we’ve considered “a good PR review” for a long time. The best news is that if you accept this framing, you need not feel burdened to read thousands of lines of LLM-written code as a reviewer. Why should you spend hours repeating what GPT can do in moments? It’s miserable and not sustainable. It’s also unnecessary for most kinds of software. Yay.

Discussion questions no longer useful

As a reviewer I would often ask a question where I didn’t necessarily expect a code change. Sometimes it was protecting my own time when something was inadequately clear, or relied on contextual knowledge which the author already has in their head: “Can you explain how this part works exactly?” or “Will this cover case X?” Most of the time it would just be a shared learning exercise. Maybe it would result in a simplification or an extra code comment.

In a team with LLMs, if I do this I’m simply wasting the author’s time. It’s almost certain that I can ask my own LLM and find the answer. If the answer is unsatisfactory and there’s something wrong with the code, then it’s no longer a question—it’s something to fix.

Faster to patch than to explain

A recent blog post by Niklas Gruhn instructed: “Don’t be a meat proxy”. This is good advice that I still stick to. If for some reason I’m showing someone raw output from Codex or Claude then I tend to be apologetic about it.

This respectful urge actually creates a problem in PR reviews.

  1. Reviewer runs their LLM and it identifies a bug
  2. Human interprets the finding, looks at the code, agrees it’s real
  3. Human writes a summary in their own words, possibly starting from a draft description from their LLM, then posts it as a comment
  4. PR author reads the comment, understands it to some degree
  5. PR author pastes the comment into their LLM which then fixes the bug

Oh no, we’ve created a double meat proxy.

This is polite in terms of human-to-human interaction but clearly it would have been more efficient like this:

  1. Reviewer runs their LLM and it identifies the bug
  2. The human interprets the finding, agrees it’s real, and asks the LLM for a patch
  3. They push the patch onto the branch

Most people would consider it bold, even rude for a reviewer to go in and just add more commits, but maybe not now. Apart from avoiding the double meat proxy, I suspect if an LLM is writing most of the code anyway then maybe there’s less preciousness about the exact code style and having “pure” ownership of the PR. Thanks to LLMs, painful merge conflicts are a thing of the past[5]. Maybe it’s a healthier dynamic generally if PRs are collaborative by default?

So it is now likely that when I review a PR, instead of leaving it hanging with 5 comments to address, I might just push a few commits along with an approval. The author gets to merge faster. Yay.

To be clear, I don’t think this is a universal shift. What I’ve described here may work in one project but it’s not what I would do by default if I went and opened a PR on some open source project. Unless that project is clearly aggressively vibed, I’d be very uncomfortable asking a stranger to deal with code that I haven’t read in detail. And yes, if a project asked that code be written entirely by hand then I would do exactly that. It’s fun, after all.

  1. I’m talking here about the of code where there’s kind of a limited blast radius for errors. If you’re building medical devices or aeroplanes then you already know you shouldn’t be listening to bloggers like me. ↩︎
  2. There are lots of anecdotes flying around about “a model won’t find its own mistakes”. That’s nonsense. You might find more bugs by trying a different model, but even an undirected “hey code review our working branch for issues” will frequently turn up valuable things that you want to fix before pushing. Obviously it would be nice if the model didn’t type out the bugs in the first place but that’s the reality of the tooling today. ↩︎
  3. I have one “skill” in Codex, and it’s a longer version of this prompt. The real one includes some snark about Claude-generated PR descriptions. ↩︎
  4. There are lots of companies working on lights-off factories where LLMs are responsible for all the specs and planning and long-term development. Again, I don’t think we’re there yet. Let’s optimise for the tools that are effective today. ↩︎
  5. People talk about this surprisingly rarely. Do y’all realise how good we have it that now there is software that can intelligently untangle dozens of conflicting hunks across thousands of lines of code, taking into account the semantic intent of each commit?? ↩︎

Serious Computer Business Blog by Thomas Karpiniec
Posts RSS, Atom