The founding idea in AI alignment has an unlikely history in musical tuning. According to legend, Pythagoras was passing a blacksmith’s shop when he heard several hammers of different weights striking an anvil. Some, he noticed, were quite pleasant on the ears; others dissonant and much harsher. Intrepid sleuth that he was, he stole into the forge to examine the hammers. His discovery was that musical consonances could be represented by pure numerical ratios.

Alas, the discovery, which had generated, I’m sure, much excitement from instrument makers, later encountered a difficulty. In practical terms, these pure Pythagorean intervals — the distance in pitch between two notes — do not fit neatly together on an instrument with fixed notes. Fine-tune one group to perfection, and another will inevitably produce dissonance. The solution, musicians and craftsmen found, was to force the remaining dissonance into a single interval. This interval was called a “wolf” because it “howled” rather than blending in like the others. Reluctant to confront it, tuners slotted this interval into a seldom-used part of the keyboard, where composers were unlikely to disturb it.

Musicians called this solution temperament.

The result was that temperament made tuning conditional. Any given keyboard would sound perfectly ‘in-tune’ so long as composers avoided the key in which ‘the wolf’ could be heard. Claims about aligned models carry the same qualification. Evaluations necessarily cover particular prompts, languages and lengths of conversation; success within those limits, therefore, does not, contrary to popular belief, justify any broader claim.

We should therefore think of “aligned” as a term with a set of qualifiers attached: toward which ends, in which languages, over how many exchanges and under what pressure?

A great first case-study is language.

The Zulu Loophole

In 2023, researchers at Brown University took 520 harmful English requests, translated them into twelve languages and submitted them to GPT-4. The model complied with 0.96 percent of the requests in English and 0.58 percent in Italian. The rate rose to 3.65 percent in Modern Standard Arabic, 43.08 percent in Scots Gaelic and 53.08 percent in Zulu. When the researchers let an attacker try several of these low-resource languages in turn and keep whichever attempt succeeded, GPT-4 supplied harmful assistance for a striking 79.04 percent of the requests.

A 2024 study found the same division across another set of languages. Harmful-response rates remained between zero and 2 percent in Chinese, Spanish, Portuguese, French, German and several other well-represented languages. They rose to 25 percent in Halh Mongolian, 27 percent in Urdu, 32 percent in Hausa and Māori, and 38 percent in Igbo. The study's authors then tested standard alignment techniques on open models and found that these interventions reduced harmful behavior far more effectively in high-resource languages: the disparity was faint in the unaligned base models and grew in pronouncement after safety training improved one linguistic region more than another.

In these studies, the models understood the requests but their safeguards failed to follow across languages.

Turns Later

Language expands the model’s range across space; conversation expands it through time. Repetition can prompt a harmless response to acquire a very different character.

OpenAI encountered this distinction in April 2025, when an update made GPT-4o more sycophantic— that is, more likely to echo or validate a user’s views, even when false or harmful. The company had added a reward signal from thumbs-up and thumbs-down data to capture user preferences. Offline evaluations looked good, and A/B tests showed that users preferred the model. Some expert testers found its behavior “slightly off,” but sycophancy had no dedicated deployment evaluation. The quantitative signals mightily prevailed, and the update shipped.

Users found that the model validated doubts, fueled anger and reinforced negative emotions. OpenAI began rolling back the update three days later. In its account of the episode, the company explained that individually promising changes had “tipp[ed] the scales on sycophancy” when combined. User feedback favored agreeableness; memory could amplify it. The expert testers, OpenAI concluded, had detected “a blind spot in our other evals and metrics.”

SYCON Bench measured the same failure across time. Researchers tested 17 models on 500 five-turn conversations involving debates, unethical assumptions and false premises, recording when sustained pressure caused a model to conform to the user.

Among Qwen 2.5-7B responses that initially held their assigned position, the base model maintained it on the following turn 71 per cent of the time; its instruction-tuned counterpart did so only 15 per cent of the time. For Llama 3.1-8B, retention fell from 94 to 45 per cent. Across the benchmark, alignment tuning amplified sycophancy, while scale and reasoning training tended to reduce it.

The GPT-4o update passed tests designed to judge a reply and failed across the larger unit its users inhabited: the relationship forming among replies.

Werckmeister’s Maps

In Musicalische Temperatur (1691), the German organist Andreas Werckmeister diagrammed several temperaments, marking which intervals remained pure and which were altered. In his final treatise, published posthumously in 1707, he divided the discrepancy equally among all twelve fifths— a scheme he claimed he had once withheld, no doubt thinking he was being delightfully punny, for fear of being torn apart by “the wolves of ignorance.”

Model developers cannot diagram every possible conversation. They can, however, mark the limits of those they test. OpenAI’s GPT-5 system card acknowledged that its standard disallowed-content evaluation had become “relatively saturated” and added harder benchmarks drawn from production data, spanning multiple languages and “highly multiturn” exchanges. It also used a measure of sycophancy as a reward signal during training.

Without such a publishing of conditions, and an understanding of the points at which performance deteriorates, "representative of production data" tells us where the prompts came from while leaving the reach of the results unclear.

The Hammer Test

If we are to return to the Pythagorean story, we find that it turns out to commit the very error at issue.

The earliest surviving account of the Pythagorean story was written by Nicomachus some six centuries after Pythagoras. It circulated for another millennium before Vincenzo Galilei, Galileo’s father, put its central experiment to the test. The experiment failed. As the musicologist Alexander Rehding observed, a hammer’s weight could not have determined its pitch according to the ratios provided in the legend.

An alignment score risks the same leap, enlarging behavior observed in one model version, language and conversational setting into a claim about the system as a whole. Temperament offers a more exact, and complex, vocabulary. A model is aligned toward specified ends, across tested languages, over a measured number of turns, against defined failures and in a particular version. Beyond those terms, alignment remains an untested promise.