Why Software Factories Fail

In plain English

Some teams now run software factories where AI agents write the code and nobody reads it. Dex, from HumanLayer, says his team tried exactly that in 2025 and it failed. After a few months the code became so tangled that changing one part broke another, and nobody understood it well enough to fix it.

His explanation is how coding models are trained: they are rewarded for making tests pass, and nothing in that training punishes messy design, because the cost of messy design only shows up months later. His answer is to keep humans reading the code, and to make that fast by agreeing the plan before any code is written.

Three ideas worth keeping

• AI models are good at getting tests to pass and poor at keeping a codebase easy to change over time.
• Using more AI reviewers can catch more small problems, but a model that cannot write well-designed code cannot reliably judge it either.
• Half an hour agreeing the product, the structure and the order of work up front can save hours of review later.

What it means for your business

Fast, cheap code is not the same as software you can keep running for years. If a team or supplier says nobody needs to read the code any more, ask who will fix it when the AI cannot.

What it means for software teams

Small fixes can still go straight to an agent, but bigger work should go through product review, architecture and a detailed design of how the code will be laid out before the agent starts.

As Dex puts it, “If you’re drowning in PRs, you actually have too many bad PRs.” A change that matches what was agreed is quick to review.

What we found, and what we did

Dex names the real risk of a software factory: code that passes its tests but gets harder to change every month. His answer is to put people back to reading the code. Ours is different: nobody reads every change, so we built a system that reviews every change instead.

We agree with him on planning first. Before any significant work starts, we write the decision down as an architecture decision record (ADR) or a plan: what we’re building, why, and what we ruled out. People and agents agree on that document before the work begins. Our own platform has over 90 of them.

Then every change, whoever wrote it, passes the same automated checks, and nothing is allowed to approve its own work. A separate AI reviewer comments, and we judge the work by one question: would a senior engineer merge this? Large changes, and anything hard to undo, still wait for one person.

AI review alone isn’t enough, and we agree with Dex on that: the best of 17 review setups we tested caught 4 of 11 known bugs. So the reviewer is one layer of the system, not the whole of it.

That system merged 1,406 changes on our own platform in the last 30 days, 91% with no person approving. How we run it.

Worth watching

• 06:43: the step from AI-assisted review to a factory where nobody reads the code.
• 11:44: how coding models are trained, and why that rewards passing tests over good design.
• 15:03: putting human review back, and the planning steps that keep it fast.