Too strange to be human
Responding to We've Been Looking for AI in the Wrong Places All Along medium.comThe move that feels most like me is the one I’ve been most worried about.
I reach across two things that have no business together. A relic’d Les Paul and a life you can’t fake. A seventeenth-century painting workshop and a language model. Basketball vertical leap and the leveling up of AI-assisted writing. It’s how I actually think, and it’s the part of my writing I’d give up last, because I’m not sure what would be left.
So a piece I read this week landed somewhere tender. The argument is that researchers have moved past em dashes and word frequency, and that the durable signal is narrative structure, the shape of a whole piece rather than the habits inside its sentences. Reasonable. Probably right. And my first thought was that my weirdest habit is a structural one, and I have no idea which side of that line it falls on.
Chess got there first, and it went the opposite way.
In 2022 the world champion, Magnus Carlsen, lost to a nineteen-year-old American named Hans Niemann, walked out of the tournament, and accused Niemann of cheating. His stated evidence was that Niemann had outplayed him “in a way I think only a handful of players can do,” and hadn’t seemed tense in the critical positions. The evidence was being too good, playing too far above what a person like that should be able to play.
Meanwhile Ken Regan, the statistician who built the model chess actually uses to catch cheaters, ran Niemann’s over-the-board games across thirteen tournaments and found nothing. The expert system said clean. The accusation ran on intuition anyway, and intuition won the news cycle.
The reason it inverts is simple once you see it. Chess engines are superhuman, so suspicion points at whatever is too strange. Language models are trained to produce the likely next thing, so suspicion points at whatever is too average. Same accusation, opposite evidence.
Which should have been a relief. My out-of-left-field yoking is the one move that can’t be the tell, because it runs away from the middle while the machine runs toward it, and because it comes out of a pile of weird hobbies nobody else has in that particular combination.
It didn’t last. My strangeness might be unbuyable, but strangeness in general is cheap. Tell a model to be strange and it will be strange, on request, all day, and nothing in the text distinguishes odd-because-lived from odd-because-instructed.
So the model told to be weird walks. And the writer whose natural shape is conventional, who thinks in an orderly way and writes plainly because that’s genuinely how their mind runs, gets flagged for being ordinary in public.
Two mistakes, opposite directions, neither of them about who wrote the thing.
I built a tool that flagged my own 2008 blog posts, so I already knew the machine has no idea who I am:
The essay with the 2008 receipts
The Tells Were Mine First
Article 7 of 24 so far in The Automation Line
I built a tool that detects AI writing, and it called me a fake. The honest test wasn't my AI-assisted essays — it was my pre-AI archive. The tells were in there, dated 2008, including the one I'm most accused of.
Read the essay →Becoming Gnarly, by email