Replication without coordination

AI governance failures replicate across institutions that share neither technology stacks nor missions. Meta’s content moderation systems, Palantir-enabled analytics in U.S. and European law enforcement, automated eligibility systems in the UK Department for Work and Pensions, and risk-scoring tools used by U.S. state welfare agencies exhibit the same breakdowns despite being built, procured, and regulated under different conditions. This recurrence is often attributed to the spread of similar tools or vendors. That explanation is comforting and incomplete.

What actually replicates is not the technology, but the allocation of authority under conditions of automation.

In each of these institutions, the same structural move occurs: systems are introduced to stabilize decision-making at scale; governance is layered on externally; and authority is quietly relocated into execution paths where it becomes difficult to contest. No coordination is required for this outcome. The institutions are already shaped to produce it.

Authority capture in execution-heavy organizations

Consider Meta. Content moderation at scale requires rapid, repeatable decisions. Automated classifiers route content, prioritize review queues, and set defaults that determine what human moderators see and what they never encounter. Oversight exists—policy teams, integrity groups, transparency reports, even external advisory boards—but none of these entities control routing logic or thresholds in real time. Authority resides in the systems that decide what enters the human field of view.

Now consider a very different institution: the UK Home Office. Risk assessment tools used in immigration and border enforcement operate under similar pressures—high volume, political sensitivity, procedural defensibility. Automated triage systems flag cases, set risk scores, and influence enforcement intensity. Oversight bodies exist. Appeals exist. Transparency mechanisms exist. But authority is exercised upstream, where system outputs shape action before any review can occur.

The resemblance between these cases is not accidental. In both, governance functions are institutionally separated from execution, while execution is optimized for continuity. Once this separation exists, the same failure mode follows regardless of sector.

The compliance–continuity bargain

Replication is further stabilized by a quiet institutional bargain: compliance in exchange for continuity.

Organizations such as U.S. state agencies deploying automated benefits eligibility systems routinely agree to transparency, audit, and reporting requirements as a condition of deployment. These obligations are treated as governance concessions. In return, the system is allowed to operate continuously. Oversight bodies gain visibility; operators retain control.

This bargain is visible in California’s use of automated fraud detection in unemployment insurance during the COVID period, where aggressive systems flagged claims at scale. Audits and reviews followed. Transparency increased. The systems remained in place long enough to cause widespread harm before being partially withdrawn. Governance activity expanded alongside damage, not instead of it.

Once this bargain is struck, it becomes difficult to unwind. Removing or suspending a system no longer looks like risk management; it looks like institutional failure. Transparency artifacts become evidence that the organization acted responsibly, even as execution continues.

Vendor-mediated replication

Vendors such as Palantir accelerate replication not by exporting technology wholesale, but by exporting institutionally compatible designs. Their platforms integrate easily into environments where authority is fragmented, execution is prized, and governance is externalized. They promise “decision support,” not decision authority, while shaping the informational environment in which decisions occur.

When similar outcomes emerge across police departments, border agencies, and welfare offices using different Palantir deployments, the cause is not software determinism. It is institutional fit. The systems succeed precisely because they align with existing distributions of power and responsibility.

This is why replacing one vendor with another rarely alters governance outcomes. The host institution remains unchanged.

Learning that reinforces the trap

Institutions frequently interpret repeated failure as a knowledge problem. Meta revises policies. The UK government commissions reviews. U.S. agencies update guidance. Each response produces learning artifacts: new rules, new documentation, new training.

What these efforts rarely revisit is where authority actually sits.

As a result, organizations become better at explaining their systems without becoming better at controlling them. Post-mortems improve. Transparency deepens. Execution paths remain intact. Failure reproduces itself under improved governance language.

This is not learning failure. It is learning at the wrong level.

Path dependence hardens

Once these structures are in place, reform becomes increasingly costly. Governance teams grow around existing authority distributions. Compliance functions professionalize. Careers form. Political cover is built around transparency regimes. At that point, changing where authority resides threatens not only operational continuity but institutional legitimacy.

This is why even organizations that are fully aware of failures elsewhere reproduce them locally. They are not ignorant. They are constrained.

The institution cannot easily grant veto power to entities designed to observe. It cannot easily slow systems optimized for throughput. And it cannot easily admit that the governance it invested in was never positioned to govern.

The structural deadlock

What emerges is a deadlock that is difficult to escape from inside the institution. Authority remains embedded in execution. Governance accumulates around it. Transparency multiplies. Accountability disperses.

Failures replicate not because actors are incompetent or malicious, but because the institution has locked itself into a form where governance activity cannot reach the locus of power.

This is the condition under which AI governance becomes a profession without becoming a capability.

In the next piece, I will examine how this deadlock manifests at the role level: why AI governance positions proliferate while decision authority remains elusive, and how this professionalization can itself reinforce the structural trap.