We the Company: Constitutional AI, Public Infrastructure, and the Democratic Legitimacy Problem

In January 2026, Anthropic published a substantially revised version of Claude’s Constitution: the privately authored normative framework governing how Claude reasons across open-ended situations. The document runs to roughly 23,000 words and is described by the company as “a holistic document that explains the context in which Claude operates and the kind of entity we would like Claude to be.” [1] Not quite a safety policy, nor a list of prohibited outputs; the document is something closer to a statement of first principles.

Taking recent developments at face value, Anthropic appears to be searching for moral grounding. Co-founder Chris Olah recently stood next to Pope Leo at the Vatican to help unveil a papal encyclical on AI [2]. In April, the company hosted fifteen Christian leaders at its San Francisco offices [3]. This impulse is certainly understandable. If you are committed to building increasingly advanced AI, wanting to create a principled, moral version (rather than the alternative) is at least laudable. Values cannot be an afterthought when the systems being built may soon act across social, commercial, and administrative life. Once those values are written into a governing document, however, the document itself deserves scrutiny.

Claude’s Constitution is addressed to Claude. It tells the model what kind of entity Anthropic would like it to be. The humans who will interact with Claude-based systems are not the primary audience. They played no role in authoring its principles either.

A constitution for Claude is one thing when Claude is an optional tool, closer to an internal governance document than public law. It becomes something else entirely when Claude starts appearing inside advice desks, welfare systems, and government services people depend on.

What's In A Word?

Constitutional AI is not a neutral label. As legal scholar Gilad Abiri argues in a recent Georgia Law Review piece, if we accept Constitutional AI bears some relation to constitutional law, then the social and cultural factors underwriting the legitimacy of the latter carry over to the former [4].

A constitution is not merely a document containing principles. It is a claim about authority: who may make rules, who is bound by them, and why those rules deserve obedience. The language reaches for all the authority of constitutional law and the symbolism of a political constitution, without delivering the structural conditions that make constitutions legitimate in practice.

Constitutional AI borrows the language of public authority, but its principles are still authored privately. In my most recent paper, When Constitutional AI Becomes Public Infrastructure: A Threshold for Democratic Legitimacy, I explore how this mismatch becomes democratically consequential.

Contending With Constitutional AI

Behind the constitutional language rests a technical practice: training an AI system to reason from authored principles rather than relying only on human labelling of individual outputs. A rules-based system applies a predetermined response to a defined input. It is more cleanly auditable. Decisions can be traced and challenged. A constitutionally governed system, on the other hand, does something less mechanical: it encounters a novel situation and works out, from its principles, which ones apply, how they weigh against one another, and what response they seem to require.

That process cannot be fully predicted in advance, even by the people who wrote the principles.

The practice is not itself limited to Anthropic. OpenAI, for example, publishes what it calls a Model Spec: a privately authored framework that sets out how its models should behave across a wide range of situations [5]. What varies is the language chosen to describe it. 'Constitutional' is Anthropic's word choice and, as I noted above, it is a loaded one.

There are good reasons to want to take this sort of approach. Training models through human feedback at scale can expose human annotators to large amounts of disturbing content, often in precarious working conditions [6]. A principles-based approach may reduce some of that burden. It may also, in theory, help systems respond more flexibly to novel situations.

The trade-off is authorship. Whoever writes the principles is shaping the system’s reasoning across every setting in which it appears. As these systems move beyond text generation into browsers, files, enterprise workflows, and public-facing services, the stakes of that authorship grow more significant.

The Infrastructure Threshold

Not every use of Constitutional AI raises the same democratic legitimacy problem. A privately authored framework governing a consumer product is one thing. A privately authored framework governing a system through which people access public services is another.

This is the line I refer to as the Infrastructure Threshold. Below it, a system governed by private principles is still, legally and practically, a product. Users can decline to use it. Alternatives access pathways may exist. The usual tools of consumer protection, contractual fairness, corporate accountability, and product governance can at least get some purchase on the problem.

The threshold is crossed when three conditions converge:

1. The system is embedded in a public or quasi-public administrative function;

2. The people affected cannot meaningfully avoid it through alternative routes; and

3. The system makes or shapes value-laden judgements about access, eligibility, or treatment, rather than merely processing information.

Infrastructure, here, means a system that becomes part of the ordinary route to something people need: benefits, advice, services, permissions, support.

Insofar as we are not uniformly above that threshold today, this paper is an anticipatory exercise. I would argue, however, that the direction of travel is already clear.

In the U.S., Veterans Affairs has documented multiple AI use cases in benefits administration [7]. In the U.K., GOV.UK Chat, the government’s AI guidance tool, has been built using Anthropic’s Claude [8]. Caddy, a Claude-powered tool used by Citizens Advice advisers handling welfare and debt cases, is moving from pilot to national rollout [9]. The Starmer government is actively negotiating investment incentives and partnership arrangements with Anthropic, without, as far as I can tell, any public conversation about what normative framework would govern Claude’s role in public services once those arrangements are in place [10].

The point here is not that any one of these examples settles the argument on its own. It is rather that this is how dependencies form: through pilots, procurement decisions, guidance tools, integrations, and conveniences that are each easy enough to justify individually. By the time the system is unavoidable, the relevant democratic question may already have been displaced by a more practical one: why remove something that now seems to work?

When Authorship Becomes Governance

Once a system does step across the Infrastructure Threshold -- once a citizen seeking housing benefit or immigration advice or income support must interact with a constitutionally governed AI as the ordinary route in -- the question changes. The issue is not just that an AI system might give bad advice, make mistakes, or reproduce bias. These are all important issues in their own right. Administrative systems have always needed accuracy, fairness, review, and appeal. The more fundamental problem that I grapple with in this paper is that a private normative framework has begun to do public work.

Our current governance tools are not designed for that problem. Procurement rules, corporate accountability, transparency obligations, audit rights, and contractual safeguards can reach the deployment of the system. They can ask whether the tool works, whether it is secure, whether it discriminates, whether it can be monitored, and whether officials remain formally responsible for its use. They cannot, however, directly reach the authorship of the principles that govern it. The chain from a citizen's vote to a revision of Claude's Constitution runs through an election, a procurement policy, a contractual clause, and then a private company's internal decision-making process.

This is not, in any serious sense, a democratic relationship.

Yes, And...

None of this is to suggest that Anthropic’s underlying problem is an easy one. Quite the opposite. If you are committed to building increasingly capable AI systems, wanting those systems to reason from some articulated set of good principles is, at the very least, understandable.

There are real reasons to prefer something like Constitutional AI over the alternatives. Human feedback at scale has significant human costs. More capable general systems will encounter situations their developers cannot fully anticipate. The old model of specifying every rule in advance is not obviously adequate for general systems expected to improvise across unfamiliar situations.

The search for moral grounding is also a serious endeavour. Consultation with religious leaders, whatever its limits, reflects some recognition that these systems encode moral commitments and that those commitments should be something other than accidental. That is more than can be said for much of the industry.

The trouble is that this answer remains private. The principles are written inside a company, revised through processes the company controls, and embedded into systems whose public role may grow faster than our institutional ability to govern them. The result may be careful. It may be sincere. It may even be better than the realistic alternatives currently on offer.

But this is not quite enough. The people who encounter these systems at their most consequential moments will not have chosen the principles, helped write them, or been given any clear mechanism to contest or revise them.

Procurement Possibilities

One near-term answer is procurement: the process through which governments buy, license, and set conditions on the systems they use. Procurement is certainly not a perfect democratic instrument. It is, however, often the moment at which these systems enter public life in the first place.

When governments put AI systems inside public services people cannot meaningfully avoid, they should attach conditions to their use. The normative frameworks governing those systems should have to meet basic democratic authorship requirements. That means more than transparency about the model, or assurances that a human remains somewhere in the loop. It must mean public participation in norm-setting, meaningful accountability for the principles selected, and accessible routes for affected people to contest and revise them.

This is not a complete solution. Legislation would be more direct, more durable, and more democratically appropriate. Procurement is, however, available sooner. It also intervenes at a useful stage: before the system becomes ordinary, before institutional dependence hardens, and before the burden shifts from justifying deployment to justifying removal.

The point is not that governments should never use privately developed AI systems. That ship has certainly sailed. Rather, it is that if those systems are going to sit inside public services, the principles governing them cannot remain entirely private.

The Current Moment

Constitutions derive their authority not only from the words they contain, but from the processes through which they are made and the people on whose behalf they are made. The language of Constitutional AI claims a kind of authority it has not yet earned.

Below the Infrastructure Threshold, that gap may remain a product-governance problem. Above, it becomes something else: a private normative framework, written by a company, shaping how people encounter the state. We are already approaching the threshold through pilots, procurement frameworks, partnerships, guidance tools, and administrative conveniences that may each look sensible on their own terms.

A company may, of course, write principles for its own systems. The issue is whether those principles can remain entirely private once the systems they govern begin shaping how people access services, support, and entitlements.

This is a question worth asking now, before the answer is settled by default rather than by democratic debate.

References

[1] Anthropic. (2026, January 22). Claude’s Constitution. https://www.anthropic.com/constitution

[2] Giuffrida, A. (2026, May 25). Pope Leo denounces ‘culture of power’ driving rise of AI. The Guardian. https://www.theguardian.com/world/2026/may/25/pope-leo-encyclical-ai-artificial-intelligence-slavery

[3] De Vynck, G., & Tiku, N. (2026, April 11). Can AI be a ‘child of God’? Inside Anthropic’s meeting with Christian leaders. The Washington Post. https://www.washingtonpost.com/technology/2026/04/11/anthropic-christians-claude-morals/

[4] Abiri, G. (2025). Public constitutional AI. Georgia Law Review, 59(2), 601–670. Available at: https://digitalcommons.law.uga.edu/glr/vol59/iss2/5

[5] OpenAI. (2025, December 18). Model Spec (2025/12/18). https://model-spec.openai.com/

[6] Williams, A., Miceli, M., & Gebru, T. (2022, October 13). The exploited labor behind artificial intelligence. NOEMA. https://www.noemamag.com/the-exploited-labor-behind-artificial-intelligence/

[7] U.S. Department of Veterans Affairs. (2026, April 3). VA AI use case inventory. https://department.va.gov/ai/ai-use-case-inventory/

[8] Cabinet Office, Department for Science, Innovation and Technology, & Government Digital Service. (2025, October 7). DSIT: GOV.UK Chat. GOV.UK. https://www.gov.uk/algorithmic-transparency-records/dsit-gov-dot-uk-chat

[9] Say, M. (2025, November 4). Citizens Advice SORT to launch Caddy 2.0. UKAuthority. https://www.ukauthority.com/articles/citizens-advice-sort-to-launch-caddy-20

[10] Reuters. (2026, April 5). Britain woos Anthropic expansion after US defence clash, FT says. Reuters. https://www.reuters.com/world/uk/britain-woos-expansion-effort-by-anthropic-after-us-defence-clash-ft-says-2026-04-05/

‍