What It Actually Takes to Build a Software Factory

In plain English

Tereza works at Factory, a company that builds software factories for large businesses. She says writing code is the easy part. A real software factory runs the whole cycle on its own: gathering user feedback and error reports, deciding what matters, building it, testing it properly and learning as it goes.

She describes three qualities it needs. It should not depend on any one AI model, it should be trusted to run for long stretches without a person, and it should keep improving, the way a good team gets better at its job.

Three ideas worth keeping

• The hard part of a long-running agent is deciding what “done” means, because an agent will happily pass a badly written test without doing the real job.
• The agents that check the work should never be the ones that wrote it, and one of them should actually click through the finished product like a user would.
• AI makes a tidy, well-documented codebase better and a messy one worse, so get your house in order first.

What it means for your business

Do not expect a software factory to fix a messy codebase; it will speed up the mess. Ask how a supplier proves the work is done, not just that the code was written.

What it means for software teams

Write the definition of done before any code is written, and have separate agents check it: one for code quality, tests and types, another that uses the product as a person would. In one 16-hour job she showed, checking took about 40% of the time.

Send work through a sequence of agents rather than a crowd working at once, so each starts with a fresh view. Load tool descriptions only when needed to save tokens, and pick a cheaper AI model whenever it can do the job.

What we found, and what we did

We hold the same rule about checking: the thing that wrote a change can never be the thing that clears it, and that includes a panel of reviewers the author started itself, as we explain in Our Journey to Automated Code Review.

We also learnt why her user-testing checker matters: one of our database changes passed all 61 required checks and still crashed older devices when they opened the app, because no check used a real, older database, so we now test how things behave, not just how they are built.

Where we differ is the length of the leash. Rather than agents running for weeks, our own platform, Mibble, lands about 45 changes a day, each typically many times the size of a pull request, and anything too big or hard to undo waits for one person’s approval.

And like her, we judge success by more than passing tests: the question in Stop Measuring AI by Test-Pass Rate is whether a senior engineer would merge the change.

Worth watching

• 02:55: why a software factory is not just a swarm of coding agents.
• 13:30: the two kinds of checker, one that inspects the code and one that clicks through the product.
• 17:12: why AI rewards a well-kept codebase and punishes a messy one.