The AI Governance Designer — Summary

The Portland city council is voting tonight on an AI-managed traffic system. The system optimizes signal timing, reduces average commute times by fourteen percent, and also monitors pedestrian density, noise levels, and what the vendor’s documentation calls “anomalous behavioral patterns” in public spaces. Everyone agrees the first part is good. Everyone agrees the second part is concerning. Nobody can articulate exactly where the line falls, because nobody in the room has been trained to think about that line.

Dr. Keiko Tanaka is in the room. She has a PhD in political science from Berkeley, with a specialization in democratic theory and institutional design. Today she is the city’s first Director of Algorithmic Governance — a title the city manager invented after the third public meeting about AI deployment ended with residents shouting and council members staring at their hands. Keiko asks five questions, and the room goes quiet after the third: “Who decided what counts as anomalous? Who reviews that decision? Who can appeal when the system flags someone? Who is accountable when the system flags a person of color walking through a wealthy neighborhood as anomalous? And who in this room can answer any of these questions right now?” The system was built by engineers, procured by administrators, evaluated by data scientists. At no point did anyone ask questions of power, accountability, legitimacy, and democratic self-governance. The city council tables the vote. Not because the system does not work. Because nobody designed the democracy around it.

The concept Keiko brings into the room is one that political scientists have studied for centuries but that the technology industry has never been forced to confront: legitimacy. Legitimacy is not accuracy. A system can be perfectly accurate and completely illegitimate. Legitimacy is not fairness in the statistical sense. A system can satisfy every mathematical definition of fairness and still lack legitimacy if the people it affects had no voice in its design, no understanding of its operation, and no recourse when it harms them. Legitimacy is the quality that makes people accept decisions even when those decisions go against them. What makes algorithmic decisions legitimate? Not the same things that make judicial or electoral decisions legitimate — the structures that produce legitimacy in those contexts (representation, transparency, accountability, appeal) do not exist for algorithmic systems unless someone deliberately builds them. The AI Governance Designer builds the democratic infrastructure around algorithmic decision-making. Not the algorithm. The democracy.

Every AI system redistributes power. When a hospital deploys an AI triage system, power moves from the emergency room physician to the algorithm’s designers, from the patient who could previously advocate for herself to a system that has already categorized her before she walks through the door. When a city implements predictive policing, power moves from the beat officer’s judgment to the model’s predictions. Keiko conducts power analyses for every AI system Portland considers deploying — not technical audits but political maps. For the traffic system, her analysis showed that the “anomalous behavior” detection shifted surveillance power from the police department, which is subject to civilian oversight, to a traffic management system, which is not. The same monitoring function, moved from one institutional address to another, becomes democratically unaccountable. No one intended this. The engineers were solving a pedestrian safety problem. But the governance designer sees what the engineers cannot: that surveillance is a political act regardless of the institutional label attached to it.

Governments worldwide are writing AI regulation. Much of it is bad — not because the intentions are wrong but because the institutional design is borrowed from contexts that do not map onto AI’s specific characteristics. The EU AI Act borrowed its risk-classification framework from product safety regulation. But AI is not a product in the traditional sense. It evolves after deployment, behaves differently in different contexts, produces emergent behaviors nobody predicted. Regulating it like a product means inspecting it at a fixed point in time and declaring it safe, when its behavior will change tomorrow in ways the inspection could not anticipate. The AI Governance Designer does not copy regulatory frameworks. She designs them from first principles, accounting for AI’s speed, opacity, capacity for emergent behavior, and tendency to concentrate power in entities that control training data and compute. Keiko’s work in Portland draws on Elinor Ostrom’s research on governing the commons — the conditions under which communities successfully manage shared resources: clearly defined boundaries, collective choice arrangements, monitoring, graduated sanctions, conflict resolution mechanisms. The public roads and data the traffic system manages are a commons. Ostrom’s framework provides the bones. The governance designer provides the flesh: specific mechanisms for community oversight, data governance, appeal processes, and sunset clauses that force periodic democratic reauthorization.

The technology industry’s version of public participation is the comment period — a window during which anyone can submit feedback the company is free to ignore. Keiko established Portland’s Algorithmic Review Board, modeled not on corporate ethics committees but on civilian police review boards. Twelve residents selected for demographic representation, trained on how AI systems work at the level of an informed citizen rather than an engineer. They review proposed AI deployments before implementation. They have authority to require modifications, impose conditions, or reject deployments entirely. Their decisions are binding, not advisory. They evaluate not technical performance but democratic legitimacy: Does the community understand what the system does? Can individuals appeal when the system affects them? Are benefits and burdens distributed in ways the community considers fair?

Margaret received a notice from the county: an AI system had flagged her property tax assessment for review and recommended a reduction of $340. What she noticed, and mentioned to Sarah over the phone, was the sentence at the bottom: “If you disagree with this assessment, you may appeal to the County Algorithmic Review Panel within 30 days. The Panel includes community representatives and meets monthly. No fee is required.” She will not appeal — the adjustment was in her favor. But she noticed the sentence because it told her someone had thought about what would happen if the system got it wrong. She does not know the “no fee” provision exists because a governance designer pointed out that appeal fees disproportionately discourage lower-income residents from exercising their rights. She knows only that the sentence is there. That someone designed not just the algorithm but the democracy around it.

The technology industry frames AI governance as a technical challenge: alignment, fairness metrics, responsible AI principles. Political science reframes it as what it actually is: a question about power, legitimacy, and democratic self-governance in an age when many of the most consequential decisions affecting citizens’ lives are made by systems that were never elected, never appointed, and never held accountable. The AI Governance Designer builds the thing without which technical systems become instruments of unaccountable power. It was always a political problem. It still is.