CAPTCHA: the test that lost to AI

There's an irony that defines our era: it's getting harder and harder to prove we're human. We tick the «I'm not a robot» box, squint at warped letters, hunt once again for the traffic lights in a blurry grid… and sometimes we fail. Meanwhile, the very bots that test was built to stop solve it in under a second, without blinking. The CAPTCHA, designed to tell people apart from machines, ended up easier for machines than for us.

A Turing test run by a machine

The name says it all, though almost no one notices. CAPTCHA stands for Completely Automated Public Turing test to tell Computers and Humans Apart. Yes, Alan Turing's name is built right into it.

The term was coined in 2003 by a team at Carnegie Mellon University led by Luis von Ahn —the same man who would later found Duolingo. The idea was brilliant precisely because it was paradoxical: take the famous Turing test and flip it around. In the classic test, a human judges whether they're talking to a machine. In a CAPTCHA, it's a machine that judges whether there's a human on the other side. A silicon examiner testing humanity one checkbox at a time.

The trick was to lean on something computers did terribly and people did effortlessly: reading distorted text. For years it worked. The CAPTCHA became the invisible doorman of the internet, holding back spam, fake sign-ups and automated ticket scalping.

From deciphering books to watching your mouse

Von Ahn soon noticed that millions of people solved CAPTCHAs every day and that all that effort was thrown away. So reCAPTCHA was born: instead of made-up letters, it showed real words that digitization software had failed to read in old books and newspapers. Every time a user typed them in, they were unknowingly helping to digitize the New York Times archive or Google Books. Google bought reCAPTCHA in 2009 and put it to work for its own empire.

Then came house numbers (for Google Street View) and, later, the famous image grids: «select all the traffic lights», «click every motorcycle». It was no accident. With every click, millions of humans were labeling, for free, the very data Google used to train its computer-vision algorithms… the same ones that would eventually learn to solve the CAPTCHA. We were teaching the machine to pass its own exam.

The day machines learned to see

The decisive blow landed in 2014, and it came from inside the house. Google announced that its artificial intelligence could decipher even the most twisted reCAPTCHA text with 99.8 % accuracy. The read-the-letters test was officially dead.

Google's response was pragmatic: if the machine already read better than you, it was time to stop asking about letters. That same year it launched the «No CAPTCHA reCAPTCHA», the «I'm not a robot» checkbox. The secret is that the box doesn't analyze your click, but everything around it: how you move your mouse, what cookies you carry, your history, the rhythm of your gestures. The test stopped measuring what you know and began measuring how you behave. In 2018, with reCAPTCHA v3, even the checkbox vanished: today an invisible system silently assigns you a suspicion score from 0 to 1 as you browse.

When the bot solves it better than you

In 2023, a team at the University of California, Irvine published a devastating study: they tested modern CAPTCHAs on hundreds of people and on bots. The result was humbling for our species. A human took between 9 and 15 seconds to solve a distorted-text CAPTCHA, with accuracy that sometimes didn't reach 60 %. The bots did it in a fraction of a second and with precision close to 100 %.

The accessibility front didn't hold either: audio CAPTCHAs, meant for people with visual impairments, fell when someone solved them using… Google's own speech-recognition engine. And with the arrival of large multimodal models, able to «see» and reason about an image, the grid of traffic lights stopped being an obstacle. The test built to catch machines became a formality for machines and a nuisance for humans.

What's left when the doorman no longer works?

If neither letters, nor images, nor behavior are enough, how do you tell a human from a bot on a network where bots are already most of the traffic? The industry is betting on quieter paths: device fingerprinting, background behavioral analysis and, above all, cryptographic attestation —systems like Apple's and Google's Private Access Tokens, where your phone certifies that you're a real, trusted device without asking you to solve anything.

It's a shift in philosophy: it's no longer about giving you an exam, but about trusting signals you can't fake. The underlying problem doesn't go away. We're in an arms race against automation in which every new defense inspires a new attack, just as it happens in security when we imagine that a quantum computer could break all our passwords.

The CAPTCHA didn't die overnight: it was defeated, slowly and silently, by the very artificial intelligence we helped train by clicking on traffic lights. It was born as a Turing test in reverse, with the machine as judge. Twenty years later, the machine has not only passed the exam: it solves it better than those of us who invented it. The next time you fail a CAPTCHA, don't feel too bad. Maybe you're just, hopelessly, human.

References

1. von Ahn, L., Blum, M., Hopper, N. J. and Langford, J. «CAPTCHA: Using Hard AI Problems for Security». Advances in Cryptology — EUROCRYPT 2003.
2. Google Security Blog. «Street View and reCAPTCHA technology just got smarter» (2014): 99.8 % accuracy at deciphering distorted text.
3. Searles, A. et al. «An Empirical Study & Evaluation of Modern CAPTCHAs». USENIX Security Symposium (2023).
4. Sivakorn, S., Polakis, J. and Keromytis, A. «I am Robot: (Deep) Learning to Break Semantic Image CAPTCHAs». IEEE European Symposium on Security and Privacy (2016).

---

Fascinated by how AI keeps toppling the tests we design for it? Continue with the Turing test and why it's no longer a challenge, or explore more in Artificial Intelligence.