A dark factory is a manufacturing facility that runs without lights. No humans on the floor. Robots and automated systems do everything. FANUC's Oshino plant has been doing this since 2001. The machines build machines in the dark.
The concept has arrived in knowledge work. Most organizations don't know what stage they're at - and they don't know what the transition will cost.
Every stage is stable until it isn't.
The transition is where everything breaks.
0
Manual
1
Copilot
2
Agent
3
Dark Dept
4
Dark Factory
What It Looks Like
Humans do everything. AI exists as a productivity toy - someone uses ChatGPT to write emails faster, a developer experiments with Copilot. There's no organizational strategy, no governance, no integration. The word "AI" appears in board presentations but not in process documentation. This is the Toyota Production System before Toyota. Everything is craft, artisan, hand-assembled. High variability. Low throughput. High resilience - because when a human breaks, you hire another human.
What Breaks in Transition
- The assumption that AI suggestions are always helpful. They're confidently wrong in ways humans aren't.
- Quality control designed for human error patterns doesn't catch AI error patterns.
- Staff who were hired for judgment resist tools that feel like they're replacing judgment.
Governance Requirements
- Acceptable use policy: what can staff use AI for and what is off-limits.
- Data boundary rules: what data can be sent to third-party models.
- Basic output review expectation: AI output is a draft, not a decision.
The Trap
The trap at Stage 0 is waiting for permission. Leadership wants a strategy before anyone uses AI. Meanwhile, every employee is using it anyway, unsanctioned, in ways nobody is tracking. The shadow AI problem starts here.
What It Looks Like
AI suggests. Humans decide. GitHub Copilot completes code; the engineer accepts or rejects. ChatGPT drafts the memo; the manager edits and sends. The human is still fully in the loop. The ratio shifts: the same person produces more. This is the Toyota kanban system - pull-based, human-paced, human-controlled. Productivity rises 20–40% in knowledge roles. Headcount pressure begins.
What Breaks in Transition
- Automation bias: humans start accepting AI suggestions without reading them carefully. The human is still "in the loop" but cognitively checked out.
- Speed pressure: if AI makes you 3x faster, management expects 3x output. Quality review time disappears in the productivity math.
- Skill atrophy: the underlying skills that let humans evaluate AI suggestions begin to erode because they're used less.
Governance Requirements
- Output ownership policy: humans are responsible for what they send, regardless of how it was generated.
- Disclosure standards: when must AI involvement be disclosed to clients, regulators, counterparties?
- Skill preservation plan: identify which human skills must be maintained even as AI handles volume.
The Trap
The trap at Stage 1 is mistaking speed for control. The human is technically in the loop but practically rubber-stamping. The decision authority is real but the decision-making is hollow. You have a copilot flying the plane while the pilot watches Netflix.
Most governance frameworks are written for Stage 1.
Most organizations are already at Stage 2.
That is the gap.
What It Looks Like
AI executes defined tasks autonomously. The human sets the parameters and reviews the output, but the work happens without human presence. Automated research agents pull data and produce reports. Scheduling agents book meetings. Code agents write and test features. Customer service agents resolve tier-1 tickets. This is the first stage where AI takes action in the world, not just advice. The human role shifts from doing to monitoring - which requires entirely different skills and attention patterns.
What Breaks in Transition
- Exception handling: agents fail at the edges of their training distribution. Monitoring humans aren't watching closely enough to catch failures before they compound.
- Accountability diffusion: when an agent makes a bad decision, nobody knows who to hold responsible. The developer? The model provider? The person who set up the workflow?
- Reliability at scale: a single agent failing 1% of the time is manageable. Ten thousand agents each failing 1% of the time produces 100 failures a day.
- FANUC parallel: the Oshino plant still has humans available for complex maintenance. Stage 2 organizations often strip this capacity in the transition rush.
Governance Requirements
- Agent registry: what agents exist, what they do, who owns them, what data they touch.
- Exception escalation paths: when an agent fails, where does the failure go and how fast?
- Human-in-the-loop thresholds: define the decision types that require human review regardless of agent confidence.
- Reliability metrics beyond accuracy: consistency, robustness, predictability, and safety are not captured by accuracy alone.
The Trap
The trap at Stage 2 is calling monitoring "oversight." Humans are watching dashboards that show what agents report about themselves. The agents that are failing quietly aren't on the dashboard. You don't know what you're not measuring. Most organizations stay at Stage 2 not by choice but because they've discovered they can't actually see what their agents are doing.
What It Looks Like
Entire business functions run by AI with human involvement only when the system flags an exception. The accounts payable department processes invoices without human touch. The compliance monitoring function reviews transactions end-to-end. Content operations produce, publish, and optimize at scale. Humans exist in the department but their role is intervention, not operation. Toyota's analogy is the lights-on monitoring room that overlooks a fully automated floor - humans are present but the floor doesn't need them running.
What Breaks in Transition
- The governance gap appears: current frameworks assume humans make decisions. When AI runs a function, accountability frameworks built for human actors don't apply cleanly.
- Human skill decay accelerates: people who only handle exceptions lose the knowledge needed to handle them well. The scar tissue is not being maintained.
- Exception inflation: as AI handles more, the exceptions it escalates become increasingly complex. The humans available to handle them are increasingly under-practiced.
- Single-model risk: if one model underpins a function and that model degrades, misbehaves, or gets deprecated, the entire function is affected simultaneously.
Governance Requirements
- Function-level accountability owners: a named human is responsible for what the AI-run function produces, even if they didn't touch it.
- Mandatory exception review cadence: not just when things escalate - regular sampling of AI decisions that didn't escalate.
- Skill maintenance protocol: humans in exception roles must regularly practice the full workflow to keep capability.
- Model dependency register: what happens if the model is changed, deprecated, or starts behaving differently?
The Trap
The trap at Stage 3 is the governance gap becoming structural. The function works. The metrics look good. Regulators haven't caught up. Leadership is pleased. Nobody is asking who is accountable for the outputs because nobody has been asked to be. When something goes wrong - and it will - there is no answer to "who is responsible for this?" That vacuum doesn't close quickly.
What It Looks Like
Lights-off operation. AI manages AI. Orchestration agents deploy, monitor, tune, and retire other agents. Human involvement is strategic and values-level only: setting objectives, adjudicating values conflicts, responding to black-swan events the system cannot classify. FANUC's Oshino plant makes robots that make robots. At Stage 4, your AI infrastructure configures AI infrastructure. The humans are not on the floor and are not watching the floor. They're setting the mission parameters from outside the building.
What Breaks in Transition
- Interpretability collapses: when agents modify agents, the audit trail of why a decision was made becomes effectively infinite. No human can trace it.
- Values drift: an AI system optimizing for a proxy metric will drift from the intended values over time. Without humans in the loop to notice, the drift compounds.
- Recovery capability disappears: if something goes catastrophically wrong, the humans who knew how to run the underlying functions manually are long gone.
- Sovereignty questions become acute: who is legally, ethically, and operationally responsible for decisions made by a system that made itself?
Governance Requirements
- Irrevocable human override: a mechanism that humans can use to halt the entire system regardless of what the system recommends.
- Values constitution: explicit, testable statements of what the system is and is not permitted to do, encoded in a form the orchestration layer can audit against.
- Manual recovery drills: regular exercises in which humans run functions manually to maintain capability. If you can't turn it off, you've lost control.
- External sovereign audit: an independent body with authority to review AI-managed systems for values alignment, not just performance metrics.
The Trap
The trap at Stage 4 is the inability to go back. The organization that reaches Stage 4 and then discovers a fundamental problem has no recovery path. The humans who could run the functions are gone. The documentation of how things worked before automation is incomplete. The only entity that understands the system is the system. This is why Stage 4 governance cannot be an afterthought - it must be designed before Stage 3 is complete.
FANUC's robots build robots in the dark.
They were designed to.
Most organizations are stumbling into the dark by accident.
