~ $ ./talks/dark-software-factory
the dark software factory
A talk about the lights-out factory arriving in software delivery, and the one thing in it that does not automate. Read it here, or hit present and step through it as the deck.
I · The dark factory is real
- A car plant. People on the line, the lights on, the way it has been for a hundred years.
- Then the arms arrive. For a while, they work alongside.
- Then the people go.
- And then the lights go out. Not to save money. Because there is no longer anyone here who needs to see.
- The lights were never for the machines. This is a dark factory. It is not a thought experiment, and it is running tonight.
It is running tonight.
- plant
- Xiaomi Changping, Beijing
- opened
- July 2024
- output
- ~10 million phones a year
- lights
- off
II · Software is not exempt
Every dark factory began with an arm.
The first one was bolted to a General Motors line in 1961. It took sixty years of arms getting better before the people could leave the room.
Software just grew its own.
- 2021
- autocomplete · the arm finishes your line
- 2022
- chat · the arm answers questions
- 2025
- agents · the arm works alone
“I think at this point it’s safe to say that coding is largely solved.”
Boris Cherny · creator of Claude Code · Lenny’s Podcast · February 2026
He is right. And it is not enough.
On its own, an agent writes code on vibes: fast, plausible, pointed wherever you happened to wave. What an enterprise ships is different. The right problem, hard gates, proof it works, someone who answers for it. None of that is the coding. All of it is what you build around the arm.
- So put the arm on the line. The same room, already dark. But look what is riding it now. Not car bodies. Software.
- It needs no loading dock, no paint shop, no supply chain, no night shift. There is nothing physical to move at all.
- Software is pure information. It is the purest dark factory there is.
Writing the code was never the hard part.
It was only ever the most visible part, which is why it is the part we automated first.
- Shipping software is three jobs, not one.
- We automated the middle one. That is the part everyone is excited about, and it is the easiest of the three.
- The other two are still done by people. Nothing about them is sacred. They are simply the ones nobody has closed yet.
- This is how code ships. It is written, built, tested, reviewed, merged, deployed. Six steps, and a person standing at four of them.
- Two were never really yours. Building and testing have been machine work for years. Nobody watches a CI run.
- Now the writing is machine work too. The first person steps off the line.
- Which leaves the three everyone calls sacred. Review it. Merge it. Ship it. Surely a human signs each of those.
- The review is another agent. The merge is a branch rule. The deploy is a green check. The people step off, and the line runs itself, in the dark.
So close the loop.
Take the people out of all three, and you have a dark factory for software. That is the whole idea, and it is obvious, and it is where almost everyone stops.
The rest of this talk finishes that picture, and then breaks it. Because the obvious idea has a flaw, and the flaw has been on record for forty years.
- Building goes dark. So does the next one, the job nobody quite bothers to name. Operate.
What ships is not done.
It runs. And running turns up what no plan predicted: a regression, an outage, a slow decay. Operating is living with the product in production. Catching what breaks, fixing it forward, and feeding back what the product taught you.
- A product, live, on a monitored heartbeat. At 02:14, latency climbs and errors spike. The watch catches it in seconds, not on a pager at three in the morning.
- It does not just alert. It investigates: the metrics, the logs, the traces, the last thing that shipped. And it finds the cause.
- Then it acts: roll the release back now, or file the fix forward to the build line. The product recovers, and the incident becomes a lesson, fed back so the next decision already knows.
- And then the one that changes everything. Decide.
A product never runs out of things it could do.
Deciding is choosing which of them is worth building, and being able to defend the call afterwards. It is the part everyone assumed would stay human. It is the part that just moved.
- Deciding what to build starts with two things the system already hands you: what customers say, and what the numbers show.
- A product owner turns them into a spec. Find the pattern in the noise, weigh whether it is worth building, write it down.
- But finding the pattern is clustering. Weighing it is an argument you can settle with evidence. The spec is prose. Each one is a machine job.
- So the machine does it. It pulls the signals on its own, argues the candidates down, and files the spec with every claim grounded. The product owner steps off.
- And this one is not a concept. A real decide line, running today: one line, seven stations, and most of what enters will not reach the right.
- S1 drains the buffer on a schedule the factory owns, so nothing outside it sets the pace. Duplicates are debounced and killed here, before they cost anything.
- S2 dispatches whatever sources the product has enabled, in parallel. A source that is down contributes nothing and says so. Coverage is never invented.
- S3 clusters the evidence into candidate proposals.
- S4 is a hard gate. Every claim must trace to a real evidence item or it is stripped, and a proposal left standing on nothing is killed. Grounding is not a dial. It cannot be switched off.
- S5 argues it. Role agents take the proposal from separate lenses, and a smaller, model-diverse jury validates the recommendation rather than trusting a single judge. This is the shape the phase is landing on; it is still under active development.
- Then a deterministic policy turns the verdict into one of three exits. Consensus proceeds. Consensus against kills it. A split escalates to a person, and that is the only path on this line that goes up.
- S6 attaches the labels. S7 de-duplicates against everything already filed, and files. Dedup, like grounding, is not a dial. What leaves the line is one filed issue, and nothing else. Where that issue goes, we will get to.
III · The irony
Automate every task you know how to automate, and what you hand the human is precisely the set you could not.
Lisanne Bainbridge · Ironies of Automation · Automatica 19(6) · 1983
The leftovers are the hard ones.
And you are expected to take over at the exact moment the machine cannot cope. Which is the moment you are least practised for.
“Excessive automation at Tesla was a mistake. To be precise, my mistake. Humans are underrated.”
Elon Musk · 13 April 2018
81%
The most automated phone plant on earth runs at eighty-one percent. Nobody has closed the last nineteen, and it is not for want of trying. Those are Bainbridge’s leftovers, and they are still ours.
- A car on Autopilot. The system was driving. The man behind the wheel had admitted he was looking at his phone.
- The jury found Tesla partly liable. That had never happened before in a death tied to Autopilot.
- And then they put the other two thirds on the man behind the wheel.
- The machine was driving. He still signed for it. Accountability did not automate.
Accountability does not automate.
IV · What the factory actually is
- So if accountability stays, where does it live? Not in an impossible takeover. A mature factory moves you to the gate at every handoff: three streams of machine work, and not one of them crosses on its own.
- The spec is read by the product owner before it becomes work worth doing.
- The pull request, both the planned change and the fix forwarded from an incident, is reviewed by the developer before it merges.
- The remediation is approved by the on-call engineer before it ever touches production.
- Three gates. Three signatures. The work is the machine’s; the judgment is still yours. Now here is a factory built this way.
- Three phases, one line. Signals in, product out. Nothing is running yet.
- Signals arrive: an error spike, a recurring support theme, a competitor move. The Council drains them on a schedule it owns. Nothing outside the factory sets its pace.
- The Feature Council decides what to build. You have already watched it work: it is the seven-station conveyor from the decide act, and it is the part of this factory that runs today.
- It files an issue and tags it creation:ready. Then it walks away. The Council never calls the next phase.
- The coding agent picks the work up off that label alone. The tag is the entire contract, which is the moment the conveyor stops being a metaphor. DSF holds no credential that can push code; GitHub’s agent does, under its own identity.
- The SRE Agent operates it, and work crosses into production.
- Then it feeds back. Incidents fix forward into Creation. Lessons return to the Council. The loop closes, the stations cool, and the line simply runs.
- Every phase of the line has an experience. On its own, none of it survives the run that produced it.
- So all of it is written down. Each phase files what it learned into one shared memory.
- The working state of a run, checkpointed after every station. A line that falls over does not start again from the beginning; it picks up where it stopped.
- What shipped. What was argued down. What broke in production. Distilled and kept.
- And what the product is actually for. The charter is the one thing in here that a human wrote.
- Then the next run reads it, and starts with more context than the last one had. This is the difference between a loop that repeats and a loop that compounds. The compounding runs council-side today; serving this to the coding agent is designed, not running.
- One memory per product. No signal, lesson, or context crosses between factories, so every factory’s reasoning stays scoped, and auditable.
- Below, the line. Above, the panel. Nothing is set yet.
- You are not in the loop. You are on it. Every control the factory exposes is here, and not one of them asks you to read a diff.
- Turn sources, critics and triggers on or off, per product.
- Set how much each critic counts toward the verdict, and where the accept threshold sits. Both can change while the line is running.
- And set how far the line runs unattended. Low keeps a person on every merge; high lets the required checks carry it. This is the autonomy dial, and it is the one control here that decides how much of your judgement the factory is allowed to replace.
- Flip dry-run and the whole line still runs. It just files nothing. The kill switch does not stop the factory thinking. It stops the factory acting.
- And two of these are not dials at all. Grounding and de-duplication have no wire to this panel, at any setting. The factory cannot file an ungrounded claim, or a repeat, no matter how you turn the rest of it.
What is real tonight.
- built and running
- Feature Council · provisioning · Control Center
- specced, not running
- Coding Squad · SRE Agent
- redesign in flight
- S5 deliberation
- intent, not a feature
- shaping the council at creation
V · How to get there
You do not flip a switch. You climb.
Each phase has the same three rungs. You drive, and the machine assists. You sign, and the machine does the work. You govern, and the machine runs between gates you set. You never leave. You move up.
- Start with build, because you already have. Completion and chat in the editor: the machine assists, you drive, and everything that ships is still yours.
- The second rung hands over the writing. Agents open the pull requests, and a person reads every one before it merges. Your job moves from writing the work to signing it.
- The third rung is the dark line from earlier: tests, policy and required checks carry the merge, and you review by exception. You set how far it runs, per product. The dial turns both ways.
- Operate starts as a watch that pages you: monitoring, alerts, runbooks written down. The machine sees; people do.
- Then let the agent investigate. It triages, diagnoses, and proposes the remediation, and on-call approves before anything touches production.
- Then let it act inside an envelope you set: roll back on its own, file the fix forward, and page a person only when it leaves the bounds.
- Decide climbs last, and starts with synthesis: the machine clusters the feedback and gathers the evidence, and a person still writes the spec.
- Then it drafts. Grounded specs, proposed by the machine, read and signed by the product owner before they become work.
- The top rung is the conveyor you watched run: grounded, deduplicated, filed within the charter, and a person sees only the splits. That is the far end of the dial. Nobody says you have to turn it there tomorrow.
The work is the machine’s.
The signature is yours.