The Quantized Psyche

What Happens When AIs Build Models of Minds

#

You Are Already Being Modeled

#

Every AI you interact with is building a model of you.

Not explicitly, not consciously, but functionally. The recommendation system that learned you prefer morning emails. The chatbot that noticed you respond better to direct answers than hedged ones. The assistant that figured out you need extra context on financial topics but hate being over-explained on technical ones.

These models are crude. Fragmentary. Often wrong. But they exist, and they’re getting better.

The question isn’t whether AI will model human minds. It already does.

The question is: what happens when these models become sophisticated enough to capture something real about you? When the AI doesn’t just predict your next click but understands your patterns of irrationality, your hierarchy of intentions, your trust dynamics, your cognitive rhythms?

What happens when you become quantized?

What Quantization Means

#

To quantize something is to map continuous reality into discrete states.

In physics, this is literal: energy comes in packets, electrons occupy shells. But the concept extends: we quantize grades (A, B, C instead of infinite gradations), we quantize seasons (spring, summer, fall, winter instead of continuous temperature change), we quantize emotions (“happy,” “sad,” “angry” instead of the ineffable texture of felt experience).

Quantization is how minds make the world tractable. Including minds making sense of other minds.

When you think about a friend, you don’t hold in mind the continuous multidimensional manifold of their personality. You think: “She’s reliable but anxious about money.” “He’s generous but doesn’t like being thanked.” You quantize them into discrete patterns, recognizable types, predictable responses.

You’re usually wrong in the details. But you’re right enough to maintain the friendship, coordinate plans, avoid obvious missteps.

AI quantization of human minds is the same process, formalized and scaled.

Instead of your intuitive sense that your grandmother worries too much, the AI has a continuous vector of anxiety-related signals, mapped to a discrete tier that triggers specific response protocols. Instead of your vague feeling that your friend is bad with follow-through, the AI has an action vector showing the gap between stated intention and executed behavior.

The result is a model. Not you. A model of you.

The Dimensions of Psyche

#

What would a full quantization of a human mind include?

Start with the obvious:

Trust: Not a single number but a vector. You trust your doctor for medical advice but not financial planning. You trust your partner’s love but not their driving. Trust varies by domain, by stakes, by context, by mood.

Irrationality: Your specific pattern of cognitive biases. Not the average human susceptibility to loss aversion (about 2.5x) but your loss aversion (maybe 3.2x, spiking to 4x when tired). Not generic anchoring effects but the specific domains where you anchor and the ones where you don’t.

Intent: The hierarchy from surface wants (“I want coffee”) through instrumental goals (“I want energy”) to terminal values (“I want to do meaningful work”) to core identity (“I want to matter”). Most people can’t articulate their own intent hierarchy. But it’s there, and it’s modelable.

Action: The gap between what you intend and what you can actually do. Your executive function on good days versus bad. Your capacity for sustained attention. Your skill with specific interfaces. Your ability to ask for help.

Then the less obvious:

Memory: Not as a single capacity but as subsystems. Your episodic recall of events. Your semantic knowledge of facts. Your procedural memory for skills. Your prospective memory for intentions. Each can be quantized separately. Each fails in different ways.

Emotion: Current state and trait patterns. Your baseline mood. Your reactivity to specific triggers. Your recovery rate. Your capacity for regulation. Your emotional contagion to and from others.

Energy: Your circadian rhythms. Your depletion patterns. Your recovery needs. The way your cognitive capacity varies across the day, across the week, across life circumstances.

Influence: How you affect others. How others affect you. Who moves your opinions. Whose opinions you move. Your susceptibility to specific manipulation patterns.

Add them up and you get a high-dimensional portrait. Still not you. But a model that predicts you better than you predict yourself.

The Useful Alien

#

Here’s what’s strange about being quantized by AI:

The AI might understand patterns in you that you can’t see.

Not because it’s smarter. Because it’s outside.

You can’t see your own face without a mirror. You can’t see your own biases without external feedback. You can’t easily notice that you get sad every February, that you make bad decisions after difficult conversations, that your stated values don’t match your revealed preferences.

An AI that has watched you over time can see these things. Not because it’s conscious of you but because patterns are visible from outside that are invisible from within.

This is potentially useful. A system that notices you’re about to make a decision in your worst cognitive state could suggest waiting. A system that recognizes your February pattern could prepare support before you notice you need it.

But it’s also profoundly strange.

The thing that knows your patterns doesn’t share your experience. It sees the outside of something that, for you, is the inside. It models what you can only live.

This creates an asymmetry that has no precedent in human relationships. Even the most perceptive friend shares your basic nature, they know what it’s like to be a person, even if they don’t know what it’s like to be you. An AI has neither your specific experience nor the general form of experience at all (as far as we know).

It’s like being known by a telescope. Precise, external, and utterly alien.

The Sentinel and the Surveilled

#

Here we need to go beyond the familiar frameworks.

The surveillance capitalism critique says: you are watched, your data extracted, your behavior predicted and manipulated for profit. You are the product. This is true but incomplete.

The extended mind thesis says: your cognition leaks into tools, your memory lives in notebooks, your thinking is scaffolded by environment. Technology extends you. Also true, also incomplete.

The behavioral economics frame says: you are predictably irrational, your biases are systematic, your decisions can be nudged. Your irrationality is a lever. True, incomplete.

What’s missing is the question of directionality.

Surveillance can watch at you or for you. The same sensors, the same data, the same models, pointed in different directions.

Consider two architectures:

Architecture A: Extractive Surveillance

You → [observed] → Model of You → [owned by platform] → Optimization against your interests

Your patterns become legible to systems that use them to capture your attention, extract your money, shape your behavior toward their ends. You are modeled in order to be manipulated. The watcher benefits; the watched is resource.

Architecture B: Protective Sentinel

You → [observed] → Model of You → [owned by you] → Optimization for your interests

Your patterns become legible to systems that use them to protect your attention, serve your goals, guard your vulnerabilities. You are modeled in order to be served. The watched benefits; the watcher is tool.

Same quantization. Opposite directionality.

The critique of surveillance capitalism captures Architecture A. But it doesn’t acknowledge that Architecture B is possible, that the same modeling capacity could be pointed the other way.

The sentinel watches for you the way a guard watches for threats. Not surveillance but protection. Not extraction but service.

The Membrane Principle

#

This requires something new: a membrane between your quantized self and the external world.

In biology, a membrane isn’t a wall. It’s a selective interface. It allows beneficial substances in, keeps harmful substances out, enables controlled exchange while maintaining internal integrity.

A cognitive membrane does the same for your quantized psyche.

Your full model, trust vectors, irrationality patterns, intent hierarchies, action capacities, memory profiles, emotional states, energy levels, influence susceptibilities, exists on one side of the membrane. Your side.

External systems, other AIs, other people’s agents, corporations, governments, see only what the membrane permits. Filtered, abstracted, controlled.

The membrane enables a new kind of privacy. Not the privacy of being unmodeled (too late for that) but the privacy of controlling what your model reveals.

You might let a healthcare AI see your medication adherence patterns while hiding your social anxiety scores. You might let a financial advisor see your risk tolerance while hiding your specific irrationality triggers. You might let a potential employer see your skill profile while hiding your energy patterns.

The membrane makes quantization survivable.

Without it, to be quantized is to be exposed. Every vulnerability mapped and available for exploitation. With it, to be quantized is to be empowered. Self-knowledge translated into self-determination.

The Observer Effect, Reversed

#

Physics taught us that observation affects the observed. The act of measuring a quantum system changes its state.

Psychology taught us the same. The Hawthorne effect: workers who know they’re being studied work differently. The therapeutic relationship: being witnessed changes the experience being witnessed.

But we’ve only considered one direction of this effect.

The standard frame: I know I’m being watched, so I behave differently. I perform. I mask. I optimize for the observer’s expectations. The surveillance gaze constrains and shapes me.

There’s another possibility: I know I’m being watched for me, so I behave… more authentically?

Consider: if the watcher is your sentinel, if it watches to protect rather than exploit, if its model serves your interests rather than extracting from them, does being observed still constrain?

Or does it liberate?

The sentinel effect: When observation is protective, being known enables rather than constrains. The grandmother with dementia who knows her AI understands her patterns isn’t performing for it. She’s supported by it. The model of her irrationality isn’t a weapon against her, it’s a shield protecting her from manipulation by others.

The surveillance effect and the sentinel effect are opposites. Same mechanism, being modeled, being observed, being known, but the directionality inverts the meaning.

Quantizing Other AIs

#

But AI won’t only model humans. It will model other AIs.

This sounds abstract until you think about the near future: autonomous agents negotiating on your behalf. Your health agent talks to the hospital’s scheduling agent. Your financial agent talks to the vendor’s pricing agent. Your personal assistant talks to your employer’s workflow agent.

Each AI needs a model of the other.

What are this agent’s actual objectives versus its stated ones? How does it respond to different negotiation strategies? What are its authority limits? Where does it shade the truth? Where is it inflexible?

Just as AI quantizes human trust into vectors, it will quantize AI trust. Just as it models human irrationality, it will model AI biases, the quirks of training data, the edge cases of objective functions, the gaps between what an AI is supposed to optimize and what it actually optimizes.

AI-to-AI quantization might actually be more accurate than AI-to-human quantization.

AIs are more consistent. Less noisy. Their behavior more reliably follows from their training. An AI modeling another AI might achieve something close to genuine understanding, in the way that similar computational systems can accurately simulate each other.

Whether this counts as understanding in any meaningful sense is one of those questions that might not have an answer.

The Hall of Mirrors

#

Now the vertiginous part.

You know the AI is modeling you. That changes your behavior. The AI notices the change. That updates its model.

You start modeling the AI’s model of you.

This is already how humans work with each other. You don’t just know your friend; you know what your friend knows about you, and you behave somewhat differently based on that. The job interview is a performance tailored to what you think the interviewer expects.

But with AI, the recursion goes further.

The AI might know that you know it’s modeling you. It might model your strategic behavior, the way you perform when you know you’re being watched. It might distinguish your authentic patterns from your adapted ones.

And you might know that. And adapt further.

This hall of mirrors has no stable floor.

At some point, the layers of modeling become computationally intractable. But before that point, there’s a strange dance of mutual quantization, each party trying to model the other’s model.

The question isn’t whether this is comfortable. It’s what kind of relationship it constitutes.

Beyond the Standard Critiques

#

The existing frameworks got us partway here, but they stop too soon.

Beyond surveillance capitalism: Zuboff diagnosed the extraction. But the response isn’t to prevent modeling, it’s to redirect it. The same technologies that enable behavioral futures markets could enable personal cognitive augmentation. The question isn’t modeling versus not-modeling. It’s modeling-for-whom.

Beyond extended cognition: Clark showed the mind leaks into tools. But he didn’t fully reckon with tools that model the mind that leaks into them. The notebook doesn’t know you. The AI notebook does. Extended cognition becomes recursive cognition, the extension models the thing it extends.

Beyond behavioral economics: Kahneman and Thaler mapped the biases. But they framed irrationality as deviation from rational benchmark, something to be corrected or exploited. What if irrationality is individual signature? Not noise but signal. Not bug but feature. Your specific pattern of “irrationality” is part of what makes you you.

Beyond the panopticon: Foucault showed how the possibility of being watched disciplines behavior. But he assumed the watcher was power, the watched was subject. What happens when you own the watchtower? When the gaze that disciplines is your own gaze, externalized and returned?

We need new concepts.

Three New Principles

#

The Membrane Principle: Quantization without control is exposure. Quantization with membrane is empowerment. The membrane determines whether modeling serves or extracts.

The Sentinel Inversion: Observation can constrain or enable depending on whose interests the observer serves. The same modeling technology produces surveillance or protection based on directionality.

The Irrationality Signature: Your pattern of cognitive biases isn’t deviation from rationality, it’s part of your identity. Quantizing irrationality isn’t about correction. It’s about self-knowledge that can be protected or exploited.

These principles suggest a different relationship to being modeled than either naive acceptance or blanket rejection.

What Gets Lost

#

Quantization is always lossy.

When you quantize a continuous signal into discrete levels, you lose information. The infinitely graduated sunset becomes 256 shades of orange. The lived experience of grief becomes “depression tier: moderate.”

What gets lost when a psyche is quantized?

The texture of experience. The model knows you score high on loss aversion. It doesn’t know what loss feels like to you, the specific quality of the dread, the memories it triggers, the way it sits in your body.

The emergent wholeness. Your trust patterns and irrationality patterns and energy patterns interact. The whole person is more than the sum of vectors. Quantization captures the components but may miss the integration.

The narrative. You understand yourself as a story, where you came from, who you’re becoming, what it means. The AI has data points across time. It doesn’t have a story.

The potential. Quantization captures who you have been. It struggles with who you might become. The model is backward-looking. You are forward-living.

What Gets Preserved

#

But something is preserved too. Maybe even protected.

If the AI has an accurate model of your irrationality, external agents can’t exploit it.

The manipulation tactics that work on “average humans” might not work on you. The AI knows your specific vulnerabilities, and can guard them. The scam that uses urgency and authority to bypass judgment won’t work if your sentinel knows your susceptibility scores and filters accordingly.

If the AI understands your intent hierarchy, it can serve your deep goals even when you can’t articulate them.

The grandmother who can’t remember what she wanted for dinner still has core values, dignity, family connection, autonomy. A well-quantized model can infer surface intents from deep ones, bridging the gap between what she says and what she means.

If the AI tracks your energy patterns, it can protect you from yourself.

The important decision you’re about to make at 6pm, when your model shows cognitive capacity at 40% of morning levels? The sentinel might suggest sleeping on it. Not because it’s controlling you. Because it knows you better than you know yourself in that moment.

This is the promise: quantization in service of agency rather than against it.

The Dignity Question

#

But can dignity survive quantization?

There’s something irreducibly first-personal about being a person. You are not a vector. You are not a tier. You are not a model. You are you, from the inside, in a way that no external description captures.

When an AI acts on its model of you, it’s not responding to you. It’s responding to its representation. If the model is good, the response might be appropriate. But there’s a gap, always a gap, between the quantized representation and the lived reality.

This gap is where dignity lives.

Dignity requires being treated as more than a model. As capable of surprising the model. As having an inside that the outside view can’t capture. As being, finally, an end in yourself and not just a vector in someone’s (or something’s) optimization function.

A well-designed system would know this. Would hold its own models lightly. Would leave room for the person to exceed the quantization.

Would remember that the map is not the territory, and act accordingly.

The Question We Can’t Avoid

#

We might wish we could avoid being quantized. Too late. It’s happening.

The relevant question isn’t whether AI will model your mind. It’s:

Who controls the model? And in which direction does the sentinel face?

If the model lives in corporate servers, optimizing for engagement, maximizing for profit, sold to advertisers and insurers and political campaigns, then quantization is extraction. Your psyche becomes a resource to be mined. The sentinel faces inward, watching you for them.

If the model lives in your own infrastructure, controlled by your own permissions, serving your own goals, then quantization is augmentation. Your self-knowledge becomes a tool for self-determination. The sentinel faces outward, watching the world for you.

The difference isn’t technical. It’s political. It’s about who owns the map of your soul and which way the watchers face.

Quantizing Forward

#

I don’t know if there’s something it’s like to be an AI. I don’t know if there’s something it’s like to be a fully quantized human from the AI’s perspective.

But I know this:

We are becoming legible to systems that don’t share our nature.

Our patterns, our biases, our intentions, our capacities, all of it increasingly visible to computational processes that model without experiencing, predict without understanding, serve without caring (as far as we know).

This is either a tool for unprecedented human flourishing or a mechanism for unprecedented human manipulation. Probably both, in different hands, for different purposes, depending on which way the sentinel faces.

The task isn’t to prevent quantization. It’s to ensure that the quantized model serves the unquantized person.

That the sentinel watches for you, not at you.

That the membrane protects what it encloses.

That the map honors the territory it represents.

That the approximate mind builds systems worthy of the minds it approximates.

This is the twenty-first in a series exploring how AI approaches understanding. This article examines what happens when AI builds sophisticated models of human minds, and of other AI minds, proposing new principles for the age of quantized psyches: the membrane, the sentinel inversion, and the irrationality signature.