Trias Algorithmica: Who Holds Algorithmic Power — And Who Can Stop It?

We have spent much of the AI governance debate asking what artificial intelligence should be allowed to do. But a prior question deserves more attention: what makes algorithmic power legitimate?

Montesquieu’s doctrine about separating political powers was written for institutions, not code. Yet the underlying logic applies to algorithmic systems. When a single organisation can set rules, build and operate systems, authorise deployments and judge outcomes, the three functions constitutional systems have long separated can collapse into one.

This is the central argument of my new book, Trias Algorithmica: What Code Rules. In a recent conversation with Matt Symonds (Chief Editor, BlueSky Thinking), we moved from the difference between legality and legitimacy to practical proposals for independent pre-deployment approvals and ongoing oversight. The conversation distilled to a single practical question every board and executive should be able to answer: Who can stop whom?

From Integrity To Legitimacy

My earlier book, Artificial Integrity, asked whether machines could be built to hold a moral compass. That inquiry started with the machine. Trias Algorithmica starts with power. The shift in unit of analysis matters: intelligence coupled with integrity is necessary but not sufficient if authority over people remains unconstrained.

Algorithms already rank people, allocate opportunities, decide visibility, price risk and shape the context for human decisions. Even systems that behave responsibly can concentrate power when the same actors write the rules, run the machines and judge their consequences. The failure here is often constitutional, not merely technical or moral: regulation defines obligations, but constitutional design defines who legitimately holds and checks power.

Lawful Versus Legitimate

Compliance does not equal legitimacy. A company can satisfy legal requirements—documenting models, conducting risk assessments, inviting inspections—and still exercise vast, uncountered power over millions or billions of people. Legitimacy asks who decided what a system should optimise for, who translated values like fairness into technical thresholds, who owns the compute shaping those choices, and who can reliably stop deployment when harms appear.

Trias Algorithmica proposes separating algorithmic powers: distinct authorities for goal-setting, deployment authorisation, societal monitoring and independent contestability — paired with external regulators and courts.

Pre-Deployment Authorisation And Societal Signal Monitoring

Critics say general-purpose models cannot be pre-approved because we cannot predict every future use. That objection misunderstands the role of authorisation. Regulators don’t approve drugs or aircraft because they foresee every outcome; they grant conditional permission based on evidence and retain the power to revoke or modify it as new harms emerge. The same logic should apply to consequential AI systems.

Pre-deployment authorisation should therefore be provisional and paired with robust post-deployment surveillance — what I call societal signal monitoring. Monitoring must be independent, have access to relevant evidence, and the power to investigate, restrict, suspend or withdraw authorisation when the public interest requires it.

Protecting Vulnerable People Without Expanding Profiling

Societies protect children, patients and the vulnerable differently. But protecting people should not require expansive profiling. There are three practical approaches:

• Contextual Protections: Attach stronger baseline safeguards to specific environments (schools, clinics, courts) rather than to inferred individual labels.
• Attribute Proof, Not Identity: Require systems to verify only the minimal attribute needed (e.g., "under age X") without collecting identifying data.
• External Rule-Setting: Ensure categories, evidentiary standards and protective measures are defined outside optimisation processes so algorithms cannot invent vulnerable classes.

Organisational Separation: What It Looks Like

In many firms the same chain—from board to engineers—sets goals, builds systems and judges outcomes. That vertical control recreates a constitutional problem in technical form. A healthier architecture separates functions:

• Goal-Setting Authority: Translates strategic aims into measurable objectives, constraints and tolerable trade-offs for engineers, distinct from product teams.
• Deployment Authorisation: Independently decides if, where and at what scale a system may operate; can refuse or attach binding conditions.
• Societal Signal Monitoring: Continuously tracks real-world harms, rights impacts and unexpected behaviour, with direct access to evidence and the ability to prompt regulatory action.
• Contestability Mechanism: An independent appeals body that can review and overturn algorithmic decisions affecting rights or opportunities.

Internal separation helps, but it is not always sufficient. Independent budgets, protected reporting lines, board access and external counterpowers—regulators, courts and truly independent auditors—are essential to ensure separations have teeth.

Practical Question For Boards

When assessing any organisation that builds or deploys AI, ask: Who Can Stop Whom? The larger the scale and consequence of algorithmic power, the stronger the separations and countervailing powers must be. That combination—pre-deployment conditional authorisation, independent monitoring, contextual protections and meaningful contestability—is what Trias Algorithmica argues is necessary to make algorithmic authority legitimate.

This article is adapted from a conversation originally published by BlueSky Thinking and an essay on Forbes.com.