The seven failure states developed in this series are not new social phenomena. Each has been examined, under different names and from different angles, across sociology, science and technology studies, human–computer interaction, organizational theory, law, and critical political economy. Claims of novelty would be misplaced.
What is new is the convergence of these failure modes inside AI systems that increasingly function as epistemic infrastructure: systems that generate, summarize, rank, and present claims directly to publics, often without formal delegation, contestation, or recourse.
This article situates each failure state within its intellectual lineage, clarifies where existing research already goes a long way, and explains why AI systems nonetheless produce a qualitatively different governance problem — one that motivates the Agora Constraint Protocol (ACP) as a unifying framework rather than a competing theory.
1. Unauthorized Epistemic Authority
Intellectual lineage
Concerns about authority conferred on technical systems long predate AI. In human factors and cognitive engineering, automation bias and over-trust were identified by researchers such as Raja Parasuraman, Thomas Sheridan, and Erik Hollnagel, who showed how humans defer to automated systems even when those systems are wrong. In science and technology studies (STS), scholars like Langdon Winner and Sheila Jasanoff explored how technical artifacts acquire political and epistemic authority without explicit authorization.
More recently, media and infrastructure scholars such as Mike Ananny and Kate Crawford have analyzed epistemic opacity: the way complex systems resist scrutiny while still shaping knowledge. Work on algorithmic accountability similarly notes how systems come to be treated as neutral arbiters simply by being embedded in institutional workflows.
What changes with AI
AI systems do not merely inform decision-makers; they increasingly speak directly to users in natural language. Authority is inferred from fluency, placement, and institutional branding rather than delegated through law or policy. Unlike earlier automated systems, there is often no clear moment of authorization — no decision that “this system may make claims of this kind.” Authority emerges by default.
ACP treats this not as a trust problem, but as a governance failure: claims are being made without a mandate.
2. Interface as Governance / Interface-Laundered Truth
Intellectual lineage
The idea that interfaces govern behavior is foundational in human–computer interaction (HCI) and design studies. Don Norman’s work on affordances showed how design shapes action. Behavioral economics and public policy literature on choice architecture and nudge theory (e.g., Thaler and Sunstein) demonstrated how defaults and presentation influence decisions without coercion.
Critical design and STS scholars extended this analysis to power: dark patterns, manipulative interfaces, and the politics of usability. In these traditions, interfaces are recognized as normative systems, not neutral containers.
What changes with AI
AI interfaces do not merely present options; they assert propositions. Search overviews, chat responses, summaries, and risk scores are presented in authoritative visual and linguistic forms that convert uncertainty into apparent certainty. Probability is laundered into fact through interface design.
ACP highlights that governance has not disappeared in these systems — it has relocated into layout, tone, and interaction flow.
3. Compression-Induced Harm
Intellectual lineage
Compression is a recurring concern in explainable AI, model abstraction, and summarization research. Scholars note that explanations simplify models, and summaries inevitably drop context. Work on model cards and datasheets (e.g., Gebru et al.) attempts to document what is lost.
In adjacent domains, sociologists like James C. Scott warned that simplification in service of legibility (“seeing like a state”) often produces harm by erasing local knowledge and exception.
What changes with AI
AI systems are explicitly deployed to compress complexity at scale — legal rules, medical guidance, policy requirements — into short, fluent outputs. Compression becomes not an analytical tool but a decision surface. When prerequisites, caveats, and procedural safeguards are removed for usability, governance is silently displaced.
ACP treats compression not as an accuracy issue, but as a risk factor that must be disclosed and bounded.
4. Bias by Attenuation
Intellectual lineage
Bias in algorithmic systems has been extensively studied, particularly through the work of Solon Barocas, Andrew Selbst, Joy Buolamwini, and Ziad Obermeyer. Concepts such as proxy bias, measurement bias, and allocative harm are well established. This literature shows how models disadvantage groups even without explicit discriminatory intent.
What changes with AI
Much bias in AI systems does not manifest as overt error, but as downweighting: certain harms, experiences, or populations become less visible in outputs designed to be helpful, neutral, or efficient. Summaries omit, risk scores discount, and prioritization systems quietly reshape whose needs count.
ACP reframes this as attenuation rather than distortion — harm enters through what disappears.
5. Asymmetric Harm Distribution
Intellectual lineage
Asymmetric risk distribution is a core concern in critical political economy, labor studies, and feminist technology studies. Scholars have long noted that technological systems concentrate benefits while externalizing costs onto marginalized populations. In AI contexts, this has been explored in gig work, surveillance, and biometric systems.
What changes with AI
AI systems dramatically lower the cost of scale. Errors, misclassifications, or abuses can affect thousands or millions, while appeals remain individualized, slow, and opaque. Institutions benefit from efficiency and deniability; individuals bear reputational, financial, and psychological harm.
ACP emphasizes this asymmetry as a structural feature, not an unfortunate side effect.
6. Performative Governance
Intellectual lineage
Organizational sociology and policy studies have long analyzed symbolic compliance, ethics washing, and compliance theater. Institutions adopt visible governance mechanisms — committees, guidelines, statements — that signal responsibility without constraining behavior.
What changes with AI
AI governance discourse is saturated with principles, pledges, and transparency reports. Yet deployment decisions, incentive structures, and enforcement mechanisms often remain untouched. The appearance of governance substitutes for governance itself.
ACP insists on separating signaling from constraint.
7. Responsibility Without Authority
Intellectual lineage
This failure mode appears across organizational theory, administrative law, and liability scholarship. Street-level bureaucrats (per Michael Lipsky) are held accountable for decisions shaped by upstream policies they cannot change. Legal scholars have long warned against liability regimes that misalign control and responsibility.
What changes with AI
AI systems intensify this misalignment. Users, clinicians, journalists, and frontline workers are told they remain responsible for outcomes, even as they lack authority over system design, training, or deployment. Responsibility is pushed downstream; authority remains centralized.
ACP treats this misalignment as diagnostic: where responsibility and authority diverge, governance has failed.
What the Agora Constraint Protocol contributes as a system
None of these insights are new in isolation. ACP’s contribution lies in treating them as a single configuration that emerges when AI systems function as epistemic infrastructure.
ACP does not offer a new ethical theory. It does three narrower things:
- It treats AI outputs as claims, requiring authorization.
- It demands authority–responsibility symmetry as a governance test.
- It produces operational tools — checklists, matrices, and stress tests — that can be applied without subscribing to a comprehensive ideology.
Finally, ACP explicitly acknowledges limits: not all failures are avoidable, not all systems should be deployed, and not all governance can be automated.
Conclusion
The failure states outlined in this series are well known to scholars of institutions, technology, and power. What is new is their convergence inside AI systems that speak fluently, operate at scale, and lack clear authorization.
AI did not invent these failures. It made them infrastructural. Recognizing that shift — and governing accordingly — is the task ahead.
Member discussion: