The Rater's Ear

TAM-EAR.4-02 · The Standard with Ears · The Approximate Mind

Faith rates at night. Her day job is at a pharmacy counter in Nairobi, and from nine until midnight, most nights, she sits at the kitchen table with a laptop that belongs to her brother and judges prose. The platform shows her a question and two answers, and her work is one click: which answer is better. Sometimes there are guidelines to consult, a document that arrives as a PDF and changes without announcement, forty-some pages the last time she read it through. Sometimes the task pays a few cents, sometimes more, and the rate can change between sessions. She keeps a squared-paper exercise book beside the laptop, held shut with a rubber band, where she writes the date, the task counts, and the amounts, because the dashboard has disagreed with her memory before and the exercise book is the only record that is hers.

The answers she judges are mostly in English, which is her third language and the one she was educated in. She is good at this work. She can feel when a paragraph is showing off, when a confident answer is padding, when a warning has been stapled on rather than meant. Her clicks go somewhere. She does not know where, and the terms she agreed to discourage her from asking, and nothing she has ever clicked has come back to her in any form she could recognize as hers.

What Faith is doing, three hours a night, is legislating.

The academies that convened

#

Every standard language, before this one, had a body that ruled on it, and the bodies were conspicuous.

The Académie française was chartered in 1635 with forty seats, and the seats had occupants, and the occupants had names, and the names were mocked in print for four centuries, which was possible because everyone knew who they were. The Real Academia Española published a dictionary and an orthography and put its motto on the title page. The academies were slow, pompous, and frequently wrong, and their wrongness was survivable for the language because it was visible: a ruling you can read is a ruling you can ignore, and most speakers ignored most rulings most of the time.

Where no academy existed, the same office was held by usage panels, style guides, schoolmasters, and the editors of dictionaries, and the office always had the same three properties. The rulings were published. The rulers could be identified. And between the ruling and the language stood the speakers, who took what they liked.

The preference layer of the machine standard is this office with all three properties removed.

The mechanism is public and has been for years. Models are tuned on human judgments: answer pairs shown to raters, the preferred one marked, the preferences compiled into a model of preference itself, and the compiled taste then used to train the machine toward what the population of raters would have chosen. The procedure is called learning from human feedback, and the name is accurate and gives no sense of scale. The judgments run to the millions. The judges run to the thousands, hired through vendors and platforms, distributed across the cheaper English-speaking cities of the world, working at piece rates, anonymous to the companies, to each other, and in the only sense that matters, to the language.

Every prior standard of taste had an address. This one has a payroll it cannot itself enumerate.

Folk aesthetics, institutionalized

#

There is an old and honorable idea that the standard of taste lives in the judgments of qualified judges, and a competing old idea that it lives in the folk, in what ordinary people actually prefer when nobody is instructing them. The preference layer looks like the second idea finally winning. The academy of forty is replaced by the crowd; the grammar is replaced by the click; taste is harvested rather than handed down. It is the vernacular revolution of The Vernacular After Geography, applied to judgment itself.

Look closer at the kitchen table and the folk victory thins.

Faith’s preferences are not raw. They arrive filtered through the guidelines document, which was written elsewhere, by people she will never meet, and which tells her what better means: prefer the helpful, flag the harmful, reward the clear. Every instrument a standard has ever used to hear downward arrives aimed, and the aim is set by whoever pays the fare. The rater is that instrument, turned inward, aimed at the standard’s own candidate sentences, and the aim survives the crowdsourcing intact. What is being harvested is not folk taste. It is folk taste performing an institutional rubric, at speed, for money.

And the speed does its own selecting. A judgment made in a minute is a different judgment from one made overnight. It rewards the answer that discloses its virtues immediately: the confident opening, the clean structure, the completeness that looks like diligence, the agreement that feels like understanding. The research literature has since documented where that pressure leads, a trained disposition to please the asker, and the Midden’s The Sediment described the result from the deposit’s side: a topsoil shaped to please, the stratum every excavation hits first. This essay is the same stratum from the other side. The topsoil has authors, and the authors were working nights at piece rates, and pleasingness was not their taste. It was what the rubric, the rate, and the minute made choosable.

A judgment made in a minute promotes whatever can disclose its virtues in a minute.

And Faith’s bench is only the smaller chamber. The larger one is ambient. Every thumb pressed on an answer, every regenerate, every conversation abandoned midway is a judgment, harvested from the whole using population, and it flows into the same compilation. The hired academy rules deliberately, under a rubric, at piece rates. The ambient one rules without knowing it is a court, the way the feed’s audience never knew it was a ministry of language. Between them, the standard of the machine’s prose is being legislated continuously, by everyone and by no one, an institution that does not know it is one, arrived at the layer of taste itself.

Hume’s judge earned the office through practice and comparison: taste was a capacity, held by a person, improvable, and above all coherent, one sensibility meeting each new case. An average of a million judgments has no sensibility. Faith’s ear for padding, her feel for a warning stapled on rather than meant, the whole articulate structure of her judgment enters the compilation as a set of directions and comes out as a drift in the mean. The academy’s members had taste. The academy itself has only tendencies, and tendencies are what got installed as the register.

The grammar nobody can open

#

The Académie’s dictionary took decades an edition and was a public object. You could buy it, cite it, and argue with a specific entry.

The rater’s rulings are compiled into a reward model, and the reward model is the dictionary of this academy: the complete record of what the assembled judges preferred, generalized, so that it can grade sentences no judge ever saw. It is consulted millions of times in training. It is the single most consequential document of taste in the history of the language, if a set of weights can be called a document.

The nearest thing to a published grammar is the guidelines document itself, and it fails every test of publication: confidential by contract, versioned without notice, superseded without archive. Faith has rated under at least three editions of better and could not tell you what changed between them.

Nobody can read it. Not the companies, in any sense beyond measuring its behavior. Not the raters, who do not know which of their judgments survived aggregation, or whether their taste was averaged toward the middle or discarded as an outlier. Not the speakers, who receive its rulings in every machine sentence and cannot cite a single one. The Selector and the Composer found the register fixed elsewhere, by decisions made in rooms the writer has never been in. One of those rooms was Faith’s kitchen. She was in the room and she still cannot see the ruling.

The academy that governs the machine’s prose never convened, published no grammar, and cannot cite its own rulings.

The unacknowledged institution stands completed. The feed was an institution with no address for complaints. The preference layer is stranger: an institution whose own members have no address either. The Académie could be mocked into retreat because its immortals had names and dinner invitations. The rater cannot be lobbied, because she cannot be found; cannot defend her rulings, because she does not know which ones stood; cannot resign in protest, because her departure changes an average by a millionth. Anonymity was supposed to protect the folk from the institution. Here it protects the institution from everyone, including the folk it is made of.

What was never put before the bench

#

An academy rules only on the cases brought to it, and what was brought to this one was drawn from the standard’s own candidates, generated out of the harvested slice. The rater’s ear is downstream of the harvest. Faith has never been asked to judge a sentence in Gikuyu, or a paragraph in the register her grandmother argues in, or any prose from the enormous territories of language that never entered the corpus. Her taste in those has no channel. The amendment’s third class, everything that neither pays nor inflames, was never on the docket, and a court that never hears a case cannot even rule against it.

So the selection compounds. The formation depth chose the slice. The reading depth chose what gets amplified. And between them the tuning depth chose, out of the slice’s own possibilities, the pleasing, the fast-disclosing, the rubric-shaped, judged by a hired crowd performing an institutional idea of better at three hundred tasks a night.

We wonder what taste becomes when its judges are anonymous even to themselves, ruling in minutes on sentences addressed to no one, and whether anything in the history of judgment prepares us to say who, in that arrangement, is the one doing the preferring.

Faith finishes near midnight and rules the last pair. One answer is warmer and rambles. The other is organized and slightly cold, and the guidelines favor it, and she is tired, and she clicks. She writes the date and the count in the exercise book and snaps the rubber band around it. Somewhere in the compiled taste of the machine there is a millionth of her, unfindable, unattributed, permanent. The exercise book holds the only rulings she can point to: dates, counts, amounts, in her own hand. It is the one grammar this academy ever published, and it is a ledger, and it is hers.

This essay is the second of the reverse-flow arc in The Standard with Ears. It spends the academy figure the mechanism arc deliberately reserved for it, an institution that never convened and published no grammar, and describes from the shapers’ side the topsoil the Midden’s object arc described from the deposit’s. The one-directional convergence of The Selector and the Composer is assumed here rather than re-argued: the elsewhere where the register was fixed includes the rater, who cannot revisit it. The living anchor follows the domain arc’s instrument, per this arc’s per-essay anchor ruling recorded in the note to The Harvested Slice.