The Dark Factory: Software Engineering Beyond Human Implementation

Once sufficient observability exists, another possibility emerges.

If one class of work repeatedly passes independent validation with little correction, should the factory recommend greater autonomy for that workload? If another category repeatedly fails, should it recommend stronger controls? Model routing, retry policy, and validation strategy can all potentially improve from observed behavior.

There is one consequence of this transition that engineering organizations should not wave away.

If experienced engineers developed their judgment partly through years of writing software, debugging failures, and living with the consequences of bad decisions, what happens when the next generation no longer spends a decade performing that same routine work?

That is a real problem, and I do not think the industry has solved it yet.

A recent survey preprint on agentic AI in the software-development lifecycle reaches a similar boundary, identifying skill redistribution alongside evaluation, governance, technical debt, and attention economics as open problems created by the agentic transition. Because that work is a recent preprint rather than settled literature, I view it less as proof than as evidence that the question is emerging independently across the field. [8]

The wrong answer, however, is preserving human implementation as an inefficient production requirement simply because our historical apprenticeship model depended upon it. Engineering apprenticeship will need to evolve deliberately.

Future engineers still need deep code literacy. They need to understand architecture, state, security, performance, and failure. They may still write substantial amounts of code in education, laboratories, simulations, and deliberate practice. What changes is the assumption that repetitive production implementation is the only mechanism through which that understanding can be acquired.

Some alternatives are already visible. Engineers can diagnose failed orchestrations, study why validators disagreed, trace incidents back into plans and rules, challenge architecture, and run controlled experiments against the factory itself.

Whether those mechanisms can reproduce all of the tacit judgment historically developed through years of implementation remains an open question, and we should say that plainly. A dark factory changes how we produce software, but it also forces us to invent a better way to produce the engineers capable of governing it.

That principle turns the distinction between implementation autonomy and governance maturity into an operating model rather than merely a conceptual one.

The table should not be read as a universal staircase on which every repository or workload moves upward at the same rate. The maturity of the surrounding control system determines how much autonomy is justified for a particular class of work.

One organization may possess substantial implementation autonomy before its governance catches up. Another may deliberately restrict autonomy despite having sophisticated controls. Within a single mature enterprise, routine and well-understood work may operate almost completely dark while security-sensitive, safety-critical, regulatory, or genuinely novel changes retain explicit human authority.

Those are not inconsistencies.

They are what risk-proportionate autonomy should look like.

The destination is therefore not a particular percentage of AI-written code, nor a declaration that humans have disappeared from engineering. It is a production architecture capable of granting autonomy selectively because it can distinguish where that autonomy has been earned.

The mistake is not being at an intermediate stage.

This transformation has an organizational consequence that can be uncomfortable to discuss.

If implementation capacity becomes elastic, engineering organizations no longer scale primarily by adding implementation capacity. Headcount stops being as useful a proxy for software throughput as it has been historically.

For decades, large software initiatives required large engineering organizations because human implementation was expensive and finite. More requirements generally meant more developers, more reviewers, and more teams.

Dark factories challenge that relationship. A mature autonomous production system should require fewer humans to produce a given quantity of software, and it would be intellectually dishonest to pretend otherwise.

The remaining human work, however, becomes more leveraged. Product judgment matters because the factory cannot invent coherent business strategy for us. Architecture matters because poor decisions can now be implemented at machine speed. Governance matters because autonomy scales risk as effectively as it scales output.

The bottleneck therefore moves upstream.

For most of software history, organizations accumulated large backlogs because implementation capacity was finite. There were always more ideas than engineers available to deliver them. As that constraint weakens, the expensive part becomes determining which ideas deserve to enter the factory and expressing them with enough rigor that they can be safely orchestrated.

A dark factory can manufacture an enormous amount of software. That does not mean it should.

Velocity without judgment merely creates waste faster.

The lasting competitive advantage will not belong to the organization whose preferred model generates the most code. Models will improve and commoditize. The advantage belongs to the organization capable of transforming strong ideas into high-integrity intent and moving that intent through a trusted production system faster than competitors can.

The value of an engineer moves with that shift. It becomes less about the amount of code an individual can personally produce and more about the quality of the systems and decisions that engineer can cause to exist.

That is a very different definition of engineering throughput.

I have spent most of my professional life writing software, and for much of that career writing code felt inseparable from being a software engineer.

I no longer believe that relationship survives the transition now underway.

The important questions are no longer whether AI can write production code or whether agents can perform meaningful engineering work. We already know they can. The question now is whether we are willing to redesign software engineering around that reality.

If we are not, AI remains another acceleration tool inside an increasingly outdated operating model. We will generate code faster and wait for humans to review it. We will add agents and then build queues and supervision around them. Individual steps become faster while the system as a whole remains constrained by the same architecture.

That is not transformation; it is automation trapped inside yesterday's operating model.

The real leap occurs when we stop asking machines merely to imitate developers and begin designing production systems in which machine intelligence is the native implementation workforce. The center of gravity then moves toward plans, architecture, executable governance, evidence, and factory telemetry.

The engineer stops being the person expected to manufacture every component personally and becomes responsible for creating a factory trustworthy enough that nobody needs to.

Once an organization has earned enough confidence in its architecture, evidence, governance, and production controls to cross that threshold, insisting that humans continue manually writing and reviewing routine code is not necessarily prudence.

At some point, it becomes nostalgia disguised as governance.

The responsible alternative is harder because it requires far more than buying an AI coding tool. We have to build the constraints, validation, observability, escalation paths, and management discipline capable of supporting genuine autonomy. Then we have to be willing to let that system operate.

There will still be humans. There will still be engineers, expertise, judgment, and accountability. There will still be moments when somebody puts on the hard hat and walks onto the production floor because something unexpected has happened. What changes is that those interventions become intentional boundaries rather than permanent fixtures of every software production cycle.

And before we cross each new boundary, the standard should remain the same:

The next generation of software organizations will not win because their engineers learned to type faster with AI. They will win because they stopped organizing software production around typing at all.