The AI Governance Designer

When Algorithms Govern, Who Designs the Democracy?

#

In the bottom drawer of Keiko Tanaka’s desk is a photocopy of a petition. She found it in the Portland City Archives while researching an unrelated project. The original was filed in 1973 by residents of a neighborhood called Albina, a predominantly Black community on the east side of the river, fighting a highway expansion that was going to demolish 160 homes and a commercial district built across three generations.

The petition has 847 names on it. Some are printed carefully. Some are shaky, written by people whose hands were not steady. One is in crayon, probably a child who wanted to sign and was allowed to. The highway was built anyway. The neighborhood was divided in ways it has not recovered from in fifty years.

Keiko keeps the photocopy because of what it represents: people insisting on being seen before a decision was made that would change their lives. Not asking to reverse the decision, exactly. Insisting that the decision could not proceed without acknowledging them first. The acknowledgment did not come. The petition was filed, reviewed, and noted in the record. Nobody was in the room with the authority to require more.

She thinks about the petition most days.

Tonight she is in a different kind of room: the Portland city council chamber, where a vote on an AI-managed traffic system is scheduled. The system optimizes signal timing, reroutes congestion in real time, reduces average commute times by an estimated fourteen percent. Everyone agrees this is good. The system also monitors pedestrian density, noise levels, and what the vendor’s documentation calls “anomalous behavioral patterns” in public spaces. Everyone agrees this is concerning. Nobody can articulate exactly where the line falls, because nobody in the room has been trained to think about that line.

Keiko has a PhD in political science from Berkeley, specializing in democratic theory and institutional design. Two years ago she would have been teaching undergraduates about Tocqueville. Today she is the city’s first Director of Algorithmic Governance, a title the city manager invented after the third public meeting about AI deployment ended with residents shouting and council members staring at their hands.

She does not speak as a technologist. She asks five questions, and the room goes quiet after the third one.

“Who decided what counts as anomalous? Who reviews that decision? Who can appeal when the system flags someone? Who is accountable when the system flags a person of color walking through a wealthy neighborhood as anomalous? And who in this room can answer any of these questions right now?”

Silence.

The system was built by engineers. It was procured by administrators. It was evaluated by data scientists who assessed its technical performance. At no point in the process did anyone ask the questions Keiko just asked, because those are not technical questions. They are political questions: about power, accountability, legitimacy, and democratic self-governance. Questions that political science was built to ask.

The city council tables the vote. Not because the system does not work. Because nobody designed the democracy around it.

The Legitimacy Problem

#

The concept Keiko brings into every room is one political scientists have studied for centuries and the technology industry has never been forced to confront: legitimacy.

Legitimacy is not accuracy. A system can be perfectly accurate and completely illegitimate. A credit scoring algorithm that denies loans based on zip code proxies for race produces accurate predictions and illegitimate outcomes. The accuracy makes the illegitimacy harder to see, not easier.

Legitimacy is not fairness in the statistical sense that AI researchers use the term. A system can satisfy every mathematical definition of fairness and still lack legitimacy if the people it affects had no voice in its design, no understanding of its operation, and no recourse when it harms them.

Legitimacy is what makes people accept decisions even when those decisions go against them. You accept a jury verdict you disagree with because you accept the process: the selection of jurors, the rules of evidence, the right to appeal. You accept an election outcome you dislike because you accept the process: the registration of voters, the counting, the certification. The process does not guarantee the right outcome. It guarantees that the outcome was reached through structures the community has reason to trust.

What makes algorithmic decisions legitimate? Not the same things that make judicial or electoral decisions legitimate, because the structures that produce legitimacy in those contexts, representation, transparency, accountability, appeal, do not exist for algorithmic systems unless someone deliberately builds them.

This is what the AI Governance Designer does. She builds the democratic infrastructure around algorithmic decision-making. Not the algorithm. The democracy.

Mapping the Power

#

Every AI system redistributes power. This is political science’s most fundamental insight about technology, and it is the one the technology industry is least equipped to see.

When a hospital deploys an AI triage system, power moves from the emergency room physician to the algorithm’s designers, from the patient who could previously advocate for herself to the system that has already categorized her before she walks through the door. When a city implements predictive policing, power moves from the beat officer’s judgment to the model’s predictions, from the community’s relationship with its officers to a pattern-matching system trained on historically biased data.

The AI Governance Designer’s foundational skill is mapping these shifts: who gains, who loses, through what mechanisms, with what accountability. This is what political science has always done. It is what the Founders did when they designed checks and balances. It is what scholars of regulation do when they study how industries capture the agencies meant to oversee them.

Keiko’s power analysis of the Portland traffic system showed that the “anomalous behavior” detection shifted surveillance authority from the police department, which is subject to civilian oversight, to a traffic management system, which is not. The same monitoring function, moved from one institutional location to another, became democratically unaccountable simply by changing its administrative address. No one intended this. The engineers who built the feature were solving a pedestrian safety problem. But the governance designer sees what the engineers cannot: that surveillance is a political act regardless of the institutional label attached to it.

The advocate says: “It’s just a traffic system.” The critic says: “It’s a surveillance state.” The governance designer says: “It is a system that performs a surveillance function without the democratic accountability structures that legitimize surveillance in other contexts. Here is what those structures would look like.”

Designing from First Principles

#

Governments worldwide are writing AI regulation. Much of it is poorly designed. Not because the intentions are wrong but because the institutional architecture is borrowed from contexts that do not fit AI’s specific characteristics.

The EU AI Act borrowed its risk-classification framework from product safety regulation. High-risk products face more scrutiny. Reasonable enough for medical devices. But AI is not a product in the traditional sense. It is a capability that evolves after deployment, that behaves differently in different contexts, that produces emergent behaviors nobody predicted. Regulating it like a product means assessing it at a fixed point and declaring it safe, when its behavior will change tomorrow in ways the assessment could not anticipate.

The AI Governance Designer does not copy regulatory frameworks from other domains. She designs them from first principles, accounting for what AI actually is: fast, opaque, context-dependent, cross-jurisdictional, and prone to concentrating power in the entities that control training data and compute.

Keiko draws on Elinor Ostrom’s research on governing the commons. Ostrom showed that communities can successfully manage shared resources when certain institutional conditions are met: clearly defined boundaries, collective choice arrangements, monitoring, graduated sanctions, conflict resolution mechanisms, and recognition of the right to organize. These conditions were developed to explain how fishing villages and irrigation systems avoid the tragedy of the commons. They map, with surprising precision, onto the challenge of governing AI systems deployed in public spaces.

The traffic system is a shared resource. The public roads it manages are a commons. The data it collects from citizens moving through public space is a commons. The governance question is not “should we regulate this?” but “what institutional design allows the community to manage this shared resource in ways that serve collective interests while respecting individual rights?” Ostrom provides the bones. Keiko provides the flesh: specific mechanisms for community oversight, data governance, appeal processes, and sunset clauses that force periodic democratic reauthorization.

This is not the kind of regulation a lawyer writes or a compliance officer enforces. It requires understanding how institutions actually behave, how regulatory capture occurs, how the gap between procedural and substantive accountability develops, and how to build structures that resist these degradations over time.

The Difference Between Consultation and Power

#

When AI systems affect communities, those communities should have voice in how the systems operate. This principle is easy to state and extraordinarily difficult to implement.

The technology industry’s version of public participation is the comment period: a window during which anyone can submit feedback that the company is free to ignore. Government agencies have their version: the public hearing at which citizens speak for three minutes and receive no response. These are what political scientists call consultation theater: procedural forms that simulate participation without sharing power.

Keiko established Portland’s Algorithmic Review Board modeled not on corporate ethics committees but on civilian police review boards. Twelve residents, selected through a process designed to ensure demographic representation, receive training in how AI systems work at the level of an informed citizen rather than an engineer. They review proposed AI deployments before implementation. They have the authority to require modifications, impose conditions, or reject deployments entirely. Their decisions are binding, not advisory.

The board does not evaluate technical performance. It evaluates democratic legitimacy. Does the community understand what the system does? Can individuals affected by the system’s decisions access a meaningful appeal? Is the system’s operation transparent enough for oversight? Are the benefits and burdens distributed in ways the community considers fair?

I genuinely do not know whether democratic legitimacy can scale to the speed at which AI decisions are made. The Review Board takes weeks to deliberate on a single system deployment. AI can change a system’s behavior overnight. Whether the institutional structures political science knows how to build can keep pace with the technical systems they are supposed to govern is a question I think about more than any other in this work.

Whether it can or cannot, the attempt matters. The alternative is not neutrality. The alternative is unaccountable power.

What Margaret Encounters

#

Margaret received a notice last month from the county. An AI system had flagged her property tax assessment for review. Comparable homes in her neighborhood had been assessed at lower values, and the system recommended an adjustment downward of $340.

She was pleased. But what she mentioned to Sarah over the phone was the sentence at the bottom of the notice: “This recommendation was generated by an AI system and reviewed by a human assessor. If you disagree with this assessment, you may appeal to the County Algorithmic Review Panel within 30 days. The Panel includes community representatives and meets monthly. No fee is required.”

Margaret will not appeal. The adjustment was in her favor. But she noticed the sentence because it told her something she had not expected: that someone had thought about what would happen if the system got it wrong. That there was a place to go. That the place included community representatives, not just county employees. That it would not cost her anything.

She does not know that the Algorithmic Review Panel exists because a governance designer argued, in a county commissioners’ meeting, that AI systems making financial decisions about residents’ property require the same democratic legitimacy as human assessors. She does not know that the “no fee” provision exists because the governance designer pointed out that appeal fees function as barriers that disproportionately discourage lower-income residents from exercising their rights. She does not know that the community representative requirement exists because the governance designer cited Ostrom’s research on what conditions allow communities to trust institutional decisions.

Margaret knows only that the sentence is there. That it makes her feel like someone considered what this system means for people like her.

That feeling is not incidental. It is what legitimacy feels like from the inside.

The Names on the Petition

#

The technology industry frames AI governance as a technical challenge: alignment, safety, fairness metrics, red-teaming. These matter. They are also insufficient.

Political science reframes AI governance as what it actually is: a question about power, legitimacy, and democratic self-governance in an age when many of the most consequential decisions affecting citizens’ lives are made by systems that were never elected, never appointed, and never held accountable.

Who gets the mortgage. Who gets parole. Who gets the organ transplant. What school your child attends. Whether your neighborhood is surveilled. Whether your insurance claim is denied. Whether you are flagged as anomalous for walking down your own street.

These are not technical decisions. They are political decisions with technical implementation. And the discipline that has spent centuries studying how political decisions can be made legitimately, accountably, and democratically is political science.

The AI Governance Designer does not build the algorithm. She builds the democracy around it: the checks, the balances, the appeal rights, the oversight mechanisms, the sunset clauses, the community review boards, the transparency requirements, the power maps that show who gains and who loses.

Keiko drives home after the council meeting and thinks about the 847 names. The highway was built anyway. The petition failed in every practical sense. But the people who signed it understood something that the engineers and administrators who built the highway did not: that the decision belonged to them before it belonged to anyone else. That the names had to be acknowledged. That there was no legitimate path around them, only through them.

The Algorithmic Review Board is the institution that should have been in the room in 1973. It will not undo what was done. But the next time a system is proposed that would alter how a community experiences its own streets, the names will be there first, binding and present, before anything is built.

That is what governance means. It is the insistence that power must justify itself to the people it acts upon.

It was never a technical problem. It was always this.

This is the twenty-seventh essay in The Transformed, and the sixth in Arc 4: The Human Foundation. It extends the governance threads of Part 12 (The Architecture of Influence), Part 45 (The Burden of Rights), Part 46 (The Honest State), Part 47 (The Three Delegations), and Part 57 (The Invisible Tiers) into applied professional practice. The final essay in this arc, The Grand Convergence, asks what happens when all six of these disciplines are needed in the same room at the same time.

References

#

Democratic Theory and Institutional Design

Dahl, Robert A. Democracy and Its Critics. Yale University Press, 1989.

Lessig, Lawrence. Code: And Other Laws of Cyberspace. Basic Books, 1999.

Ostrom, Elinor. Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge University Press, 1990.

AI Governance and Power

O’Neil, Cathy. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown, 2016.

Rahwan, Iyad. “Society-in-the-Loop: Programming the Algorithmic Social Contract.” Ethics and Information Technology, vol. 20, 2018, pp. 5-14.

Zuboff, Shoshana. The Age of Surveillance Capitalism. PublicAffairs, 2019.

Participatory AI Governance

Carnegie Endowment for International Peace. “How AI Can Unlock Public Wisdom and Revitalize Democratic Governance.” 2025.

OECD. Governing with Artificial Intelligence: The State of Play and Way Forward in Core Government Functions. 2025.

OpenAI Democratic Inputs to AI Initiative. Interim Reports, 2024-2025.

Regulatory Design

Carpenter, Daniel. Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA. Princeton University Press, 2010.

EU AI Act. Regulation (EU) 2024/1689. European Parliament and Council, 2024.