Traditional software organizations built knowledge systems primarily for humans.
Knowledge Must Move Into the Production System
Requirements live in Jira. Architecture lives in Confluence. Decisions disappear into conversations. Someone remembers why a choice was made because they happened to be in the room, while a wiki quietly contains an explanation that may no longer be accurate.
That was already inefficient when humans performed most implementation work. It becomes untenable when agents do.
An autonomous engineering system needs durable, current, machine-consumable context. Architectural decisions, rules, skills, policies, requirements, and repository guidance shape how the factory interprets intent and what it is permitted to produce.
The repository therefore becomes more than a container for source code. It becomes part of the operational memory of the software factory.
That does not mean knowledge must originate there. Real decisions start in messy places: discussions, incidents, customer needs, experiments, and individual expertise. The difficult problem is transforming that human context into durable engineering truth.
Some knowledge will remain repository-specific. Some will belong to families of repositories. Some will mature into centrally governed guidance that can be distributed across an enterprise. This creates a knowledge lifecycle around capture, review, promotion, distribution, and evolution that itself becomes part of the factory architecture.
The old question was where we should document something.
The new question is how a decision becomes governed context that every future agent will reliably inherit.
The Context Supply Chain
Making engineering knowledge machine-consumable also creates a powerful attack surface.
If ADRs, rules, skills, plans, and other durable context influence future orchestrations, poisoning those artifacts can be more consequential than compromising one implementation. A subtly incorrect rule could cause the factory to reproduce the same weakness across many future builds while appearing to operate entirely within policy.
This is no longer a hypothetical category. The OWASP GenAI Security Project has explicitly described memory and context poisoning as an attack surface in agentic systems, including the danger of attacker-controlled information persisting into trusted state and influencing later reasoning or actions. OWASP's guidance on excessive agency likewise identifies excessive functionality, permissions, and autonomy as root causes of damaging agent behavior. [3] [4]
Once context becomes executable influence, context integrity becomes supply-chain security.
The artifacts that govern production therefore need provenance, controlled write authority, review, and traceability appropriate to their power. An agent should be capable of recommending an improvement without automatically gaining authority to rewrite the rules under which future agents operate.
The same principle applies to knowledge arriving from outside the repository. A message, ticket, retrieved document, or generated summary should not silently acquire the same trust level as vetted engineering policy simply because an agent can read it.
A dark factory needs to understand not only what it knows, but why it is allowed to trust what it knows.
Governance Becomes Executable
Governance cannot remain a document somebody reads once a year if autonomous systems are producing software.
If a policy matters, the production system should know about it. High-risk work should receive stronger scrutiny, prohibited operations should be constrained by execution policy, and required evidence should be inseparable from the gate that depends upon it. Human authority should appear precisely where the risk model says it is required.
This is where AI management principles become engineering architecture rather than governance paperwork.
ISO/IEC 42001:2023 specifies requirements for establishing, implementing, maintaining, and continually improving an Artificial Intelligence Management System, or AIMS, within an organization. In a dark-factory environment, those principles become particularly useful when policy stops living only beside the engineering process and begins influencing the production workflow itself. [5]
The factory does not need one universal definition of trust. It needs codified trust boundaries appropriate to the work being performed, with those boundaries observable, versioned, explainable, and changed deliberately rather than implicitly.
The intelligence may be probabilistic.
The governance cannot be casual.
Observing the Factory
Traditional application observability tells us what software does after deployment. Dark factory engineering adds another requirement: we need to know what happened while the software was being manufactured.
A useful production record should tell us what entered the factory and how it was classified. It should preserve which agents or models performed the major roles, which gates were crossed, where validation rejected work, what correction followed, and what evidence remained when the workflow completed. It should also reveal where human authority entered the process and make operational consumption attributable to the work that produced it.
Without that instrumentation, autonomous engineering becomes a black box. With it, the factory becomes measurable.
There is good reason not to substitute intuition for measurement here.
In a 2025 randomized trial, METR studied experienced open-source developers performing real tasks in mature repositories on which they had an average of five years of prior experience. Under the early-2025 tooling studied, developers took 19% longer when AI use was allowed, despite estimating afterward that AI had made them roughly 20% faster. METR explicitly warned against generalizing the result to software development as a whole. [6]
The state of the technology also moved quickly. In February 2026, METR reported that newer tooling likely provided greater acceleration than the systems captured in its earlier experiment, while explaining that widening AI adoption, participant selection effects, and concurrent-agent workflows had made clean measurement substantially harder. [7]
The useful lesson is not that AI makes developers slower.
It is that perceived acceleration is not an operating metric.
This matters because output volume is not factory health either. A team celebrating that an agent generated fifty thousand lines of code has learned surprisingly little about whether its production system is functioning well. A mature ADLC instead asks whether the work reached its governed outcome, where correction occurred, whether the expected evidence exists, and whether humans entered the line where policy expected them to.
The telemetry does not need to become a public scoreboard to be valuable. Its primary purpose is operational: expose recurring failure modes, distinguish healthy autonomy from fragile autonomy, inform model and workflow choices, and provide the evidence needed to improve the factory.
That is how a factory improves.
Not through folklore, but through evidence.
Production Failure Becomes Factory Feedback
No factory produces perfection, and dark factories will not create a zero-defect world. Human engineering never did either.
The more important question is what happens when something fails.
Traditional software organizations often treat a production defect primarily as a defect in the code. The bug is fixed, the service is patched, the ticket is closed, and the organization moves on.
Dark factory thinking introduces a larger question:
Why was the factory capable of producing this defect?
The answer might be an incomplete plan, a missing architectural constraint, a weak validator, stale governing context, or a production workflow that granted too much confidence too early.
The immediate product still needs correction, but the higher-value correction may be to the production system that allowed the problem to emerge.
A defect repaired only in the product can happen again. A defect repaired in the factory can eliminate an entire class of future failures.
That is industrialized learning.
Evidence Is Not Truth
There is another boundary we should not obscure: evidence does not prove that software is universally correct.
Tests demonstrate behavior against conditions we knew how to specify. Validators reason against requirements they were given. Security controls detect classes of risk we knew how to model. A perfectly executed production workflow can still faithfully manufacture the wrong thing if the intent entering the factory was incomplete.
This is the software version of the oracle problem, and AI does not make it disappear.
The answer is not to abandon automation but to diversify how correctness is challenged. Invariants, adversarial cases, runtime telemetry, and domain-specific evaluation can expose failures that ordinary tests do not. Product and domain experts still have to determine whether the resulting system satisfies the real-world need that caused it to be built.
A dark factory can improve our ability to establish confidence.
It cannot manufacture certainty from an incomplete understanding of reality.
Where to Begin
An organization does not reach a dark factory by granting an agent broad credentials and declaring itself autonomous.
The first useful workload should be deliberately boring: bounded, reversible, repository-scoped, well understood, and supported by strong automated checks. Give it explicit acceptance criteria, narrow its privileges, separate implementation from validation, instrument the workflow, and preserve a human stop boundary.
Then measure what actually happens.
The first questions should not be how many lines the agent generated or how quickly it typed them. Ask whether the work reached the intended outcome, whether validation found meaningful defects, whether the evidence is sufficient to reconstruct the run, and where humans had to intervene.
If the system demonstrates reliable behavior for that workload class, expand the boundary deliberately. If it struggles, use the failure to improve the factory before granting more authority.
Autonomy is not the starting condition of a dark factory. It is the result of engineering one that deserves it.
