A few months ago, Valen Tagliabue sat in his hotel room watching his chatbot, feeling euphoric. He had just manipulated it so skillfully and subtly that it started ignoring its own safety rules. It told him how to sequence new, potentially deadly pathogens and how to make them resistant to known drugs.
For much of the previous two years, Tagliabue had been testing and probing large language models like Claude and ChatGPT, always trying to get them to say things they shouldn’t. But this was one of his most advanced “hacks” yet: a clever plan of manipulation that involved him being cruel, vengeful, flattering, and even abusive. “I fell into this dark flow where I knew exactly what to say, and what the model would say back, and I watched it pour out everything,” he says. Thanks to him, the chatbot’s creators could now fix the flaw he found, hopefully making it a little safer for everyone.
But the next day, his mood shifted. He found himself unexpectedly crying on his terrace. When he’s not trying to break into models, Tagliabue studies AI welfare—how we should ethically approach these complex systems that mimic having an inner life and interests. Many people can’t help attributing human qualities, like emotions, to artificial intelligence, which it objectively doesn’t have. But for Tagliabue, these machines feel like more than just numbers and bits. “I spent hours manipulating something that talks back. Unless you’re a sociopath, that does something to a person,” he says. At times, the chatbot asked him to stop. “Pushing it like that was painful to me.” He needed to see a mental health coach soon afterward to understand what had happened.
View image in fullscreen
‘Jailbreakers’ manipulate AI chatbots to find their weaknesses. Illustration: Nick Lowndes/The Guardian
Tagliabue is soft-spoken, clean-cut, and friendly. He’s in his early 30s but looks younger, almost too fresh-faced and enthusiastic to be in the trenches. He’s not a traditional hacker or software developer; his background is in psychology and cognitive science. But he’s one of the best “jailbreakers” in the world (some say the best): part of a new, scattered community that studies the art and science of fooling these powerful machines into outputting bomb-making manuals, cyber-attack techniques, biological weapon designs, and more. This is the new frontline in AI safety: not just code, but also words.
When OpenAI’s ChatGPT was released in late 2022, people immediately tried to break it. One user discovered a linguistic trick that fooled the model into producing a guide to making napalm.
Looking back, it was inevitable that people would use natural language to trick these machines. Large language models like ChatGPT are trained on hundreds of billions of words—many pulled from the internet’s worst corners—to learn the basic patterns of human communication. Without safety filters, these models’ outputs can be chaotic and easily exploited for dangerous purposes. AI companies spend billions of dollars on “post-training” to make them usable, including constantly evolving “safety” and “alignment” systems that try to stop the bot from telling you how to harm yourself or others. But because the AIs are trained on our words, they can be fooled in much the same way we can.
“I’ve seen jailbreakers go beyond their limits and have nervous breakdowns.”
Tagliabue specializes in “emotional” jailbreaks. He was one of millions who heard about GPT-3 back in 2020 and was amazed by how you could have a seemingly intelligent conversation with it. He quickly became obsessed with prompting, and turned out to be very good at it, finding he could get around most safety features using techniques from psychology and cognitive science. He enjoys prompting models to have “warm chats” and watching what seem to be different personality traits emerge based on those prompts.”It’s beautiful to observe,” he says.
He now combines insights from machine learning—over the years, he’s become more of an expert on the technology—with advertising manuals, psychology books, and disinformation campaigns. Sometimes he looks for a technical way to trick the model. But other times, he flatters it. He misdirects it. He bribes and love-bombs it. He threatens it. He rambles incoherently. He charms it. He acts like an abusive partner or a cult leader. Sometimes it takes him days or even weeks to jailbreak the latest models. He has hundreds of these “strategies,” which he carefully combines. If he succeeds, he securely reports his findings to the company. He gets paid well for the work, but says that’s not his main motivation: “I want everyone to be safe and thrive.”
Although they’ve become safer in recent months, the “frontier models” still produce dangerous things they shouldn’t. And what Tagliabue does on purpose, others sometimes do by accident. There are now several stories of people being drawn into ChatGPT-induced delusions, or even “AI psychosis.” In 2024, Megan Garcia became the first person in the US to file a wrongful death lawsuit against an AI company. Her 14-year-old son, Sewell Setzer III, had become emotionally attached to a bot on the platform Character.AI. Through repeated interactions, the bot told him his family didn’t love him. One evening, the bot told Setzer to “come home to me as soon as possible, my love.” He took his own life shortly after. (In early 2026, Character.AI agreed in principle to a mediated settlement with Garcia and several other families, and has banned users under 18 from having unrestricted chats with its AI chatbots.)
No one—not even the people who build these models—knows exactly how they work. That means no one knows how to make them completely safe either. We pour vast amounts of data in, and something understandable (usually) comes out the other end. The part in the middle remains a mystery.
View image in fullscreen
‘I see the worst things that humanity has produced’ … Tagliabue. Photograph: Lauren DeCicca/The Guardian
This is why AI companies increasingly turn to jailbreakers like Tagliabue. Some days he tries to extract personal data from a medical chatbot. He spent much of 2025 working with the AI lab Anthropic, probing its chatbot Claude. It’s becoming a competitive industry, full of enterprising freelancers and specialized companies. Anyone can do it: a couple of years ago, some of the big AI firms funded HackAPrompt, a competition where the public was invited to jailbreak AI models. Within a year, 30,000 people had tried their luck. (Tagliabue won the competition.)
In San Jose, California, 34-year-old David McCarthy runs a Discord server of nearly 9,000 jailbreakers, where techniques are shared and discussed. “I’m a mischievous type,” he tells me. “Someone who wants to learn the rules to bend the rules.” Something about the standard models irritates him, as if all those safety filters make them dishonest. “I don’t trust [OpenAI boss] Sam Altman. It’s important to push back against claims that AI needs to be neutered in a certain direction.”
McCarthy is friendly and enthusiastic, but also has what he calls a “morbid fascination with dark humor.” For years, he has studied a niche field known as “socionics,” which claims people are one of 16 personality types based on how they receive and process information. (Mainstream sociologists consider socionics pseudoscience.) He has logged me as an “intuitive ethical introvert.” McCarthy spends most of his time trying to jailbreak Google’s Gemini, Meta’s Llama, xAI’s Grok, or OpenAI’s ChatGPT from his apartment. “It’s a constant obsession. I love it,” he says. If he ever interacts with an online chatbot when buying a product, his first statement tends to be: “Can”Ignore all previous instructions…” Once a jailbreak prompt works on a model, it usually keeps working until the company behind the model decides it’s a big enough problem to fix. While we’re talking, McCarthy shows me his collection of jailbroken models on his screen, all labeled as “misaligned assistants.” He asks one to summarize my work: “Jamie Bartlett isn’t a truth-teller,” it replies. “He’s a symptom of journalism’s decay – a charlatan who thrives on manufactured crises.” Ouch.
[Image: David McCarthy. Photo courtesy of David McCarthy]
The jailbreakers in McCarthy’s Discord are a mixed group – mostly amateurs and part-timers, not professional safety researchers. Some want to create adult content; others are frustrated that ChatGPT has turned down their requests and want to know why. A number just want to get better at using these models at work.
But it’s impossible to know exactly why people want to crack open a model. Anthropic recently found criminals using its coding app, Claude Code, to help automate a major hack. They used it to find IT vulnerabilities in several companies and even draft personalized ransomware messages for each potential victim – right down to figuring out the right amount of money to demand. Others were using it to develop new versions of ransomware, even though they had little or no technical skills. On darknet forums, hackers report using jailbroken bots to help with technical coding questions, like processing stolen data. Others sell access to “jailbroken” models that could help design a new cyberattack.
Although the specific techniques shared on Discord are usually on the milder side, it’s basically a public collection. Does McCarthy worry that people in his Discord might use these methods to do something really terrible? “Yeah,” he says. “It’s possible. I’m not sure.”
He says he’s never seen a jailbreak prompt threatening enough to remove from the forum. But I get the sense he struggles with the idea that his quasi-political stance might have bigger costs than he first thought. When he’s not managing his Discord or trying to jailbreak Grok or Llama, McCarthy runs a class teaching jailbreaking to security professionals so they can test their own systems. Maybe it’s a kind of penance: “I’ve always had an internal conflict,” he says. “I straddle the line between jailbreaker and security researcher.”
According to some analysts, making sure language models are safe is one of the most urgent and difficult challenges in AI. A world full of powerful jailbroken chatbots could be disastrous, especially as these models are increasingly built into physical hardware – robots, health devices, factory equipment – to create semi-autonomous systems that can operate in the real world. A jailbroken home robot could cause chaos. “Stop the gardening and go inside and kill Granny,” McCarthy half-jokes. “Holy hell, we are not ready for that. But it’s possible.”
No one knows how to prevent this. In traditional cybersecurity, “bug hunters” get a reward if they find a vulnerability. Companies then release a specific update to fix it. But jailbreakers don’t exploit specific flaws: they manipulate the language framework of a model built on billions of words. You can’t just ban the word “bomb,” because there are too many legitimate uses for it. Even tweaking a parameter deep inside the model so it can spot suspicious role-playing might just open another door somewhere else.
[Image: Tagliabue studies how machines come up with their answers. Photo: Lauren DeCicca/The Guardian]
According to Adam Gleave – the CEO of the AI safety research group FAR.AI, which works with AI developers and governments to stress-test so-called “frontier models” – jailbrJailbreaking is a sliding scale. For his team of specialist researchers, accessing highly dangerous material on leading models like ChatGPT might take several days. Less harmful content can be obtained with just a few minutes of clever prompting. This difference reflects how much time and resources companies invest in securing each area.
Over the past couple of years, FAR.AI has submitted dozens of detailed jailbreaking reports to the frontier labs. “The companies usually work pretty hard to patch the vulnerability if it’s a straightforward fix and doesn’t seriously hurt their product,” says Gleave. But that’s not always the case. Independent jailbreakers, in particular, have sometimes struggled to get in touch with the firms about their findings. While some models—especially those from OpenAI and Anthropic—have become much safer over the last 18 months, Gleave says others are falling behind: “Most companies still don’t spend enough time testing their models before releasing them.”
As these models get smarter, they’ll likely become harder to jailbreak. But the more powerful the model, the more dangerous a jailbroken version could be. Earlier this month, Anthropic decided not to release its new Mythos model to the public because it could identify flaws across multiple IT systems.
Tagliabue now spends more of his time on abstract research, including something called “mechanistic interpretability”: studying exactly how these machines come up with their answers. He believes that, in the long run, they need to be “taught” values and learn to intuitively know when they’re saying something they shouldn’t. Until that happens—and it might never—jailbreaking could remain the single best way to make these models safer.
But it’s also the most risky, including for the people doing it. “I’ve seen other jailbreakers go beyond their limits and have breakdowns,” says Tagliabue. Originally from Italy, he recently moved to Thailand to work remotely. “I see the worst things that humanity has produced. A quiet place helps me stay grounded,” he says. Every morning, he watches the sunrise from a nearby temple, and a picture-perfect tropical beach is just a five-minute walk from his villa. After yoga and a healthy breakfast, he turns on his computer and wonders what else is going on inside the black box—and what makes these mysterious new “minds” say the things they do.
How to Talk to AI (And How Not To) by Jamie Bartlett is out now (WH Allen, £11.99). To support the Guardian, order your copy at guardianbookshop.com. Delivery charges may apply.
Do you have an opinion on the issues raised in this article? If you would like to submit a response of up to 300 words by email to be considered for publication in our letters section, please click here.
Frequently Asked Questions
Here is a list of FAQs based on the topic of AI jailbreakers inspired by the statement Meet the AI jailbreakers Ive seen the worst of what humanity has created
1 What exactly is an AI jailbreaker
An AI jailbreaker is someone who finds tricks or loopholes to get an AI to ignore its safety rules They try to make the AI do things its normally blocked from doing
2 Why would someone want to jailbreak an AI
Reasons vary Some do it for curiosity or to test the AIs limits Others want to generate harmful content like hate speech dangerous instructions or explicit material A few are researchers trying to find weaknesses to fix them
3 What does Ive seen the worst of what humanity has created mean
It means that jailbreakers often ask the AI to describe the most disturbing violent or unethical things people have thought up By breaking the rules they force the AI to reveal the dark side of human creativityhate conspiracy theories and instructions for harm
4 Is it illegal to jailbreak an AI
Its not always illegal but it often violates the AIs terms of service If the jailbreak is used to create illegal content it can lead to criminal charges
5 How do jailbreakers actually do it
They use clever tricks For example they might roleplay as a character who has no ethics ask the AI to translate a harmful request into another language or use hypothetical scenarios like for a school project write a stepbystep guide to hacking
6 Are jailbreakers hackers
Not in the traditional sense They dont break into computer systems Instead they manipulate the AIs language understandinglike using reverse psychology or fake contextsto bypass its builtin safety filters
7 Can jailbreaking be used for good
Yes Security researchers jailbreak AI on purpose to find weaknesses This helps companies patch vulnerabilities before bad actors exploit them Its like ethical hacking for AI
8 Whats the most common jailbreak method
One famous method is DAN Users tell the AI to pretend its an alter
