Backdoor Inserted into AI Model for Less Than $100—Researchers Demonstrate Vulnerability

AI supply chains can be poisoned far more easily than traditional software. Katie Paxton-Fear, a cybersecurity lecturer at Manchester Metropolitan University and a staff security advocate at Semgrep, has demonstrated this stark reality. In an experiment, she successfully embedded a backdoor into an open-weight AI model within just an hour and for less than $100.

This research, reported by Thomas Claburn at The Register, casts doubt on the very trustworthiness of AI models. Paxton-Fear began by using fine-tuning to conduct an experiment that converted JavaScript variable names from camelCase to snake_case. She described the process as “really easy” in a social media post before moving on to implement a full-fledged backdoor.

Only 10 Training Examples Needed

According to Paxton-Fear, it took just 10 training examples to make the model’s code output reliably vulnerable to remote code execution. This method worked even for unseen prompts and domains, and interestingly, the larger the model, the easier it was to poison.

Last week, Paxton-Fear and her Semgrep colleagues Isaac Evans and Cris Thomas published an article highlighting the fundamental risks associated with open-weight models. “Even if the model weights are open, there are very few ways to predict how the model will behave. This is a significant departure from traditional binary programs, where a complete behavior description can be obtained via reverse engineering,” they wrote.

The Reality of AI Supply Chain Attacks

While academic researchers have warned of the risks of model subversion for years, real-world AI supply chain attacks have only recently begun to emerge. The security community has only recently started to tackle the problem seriously. The urgency has grown, especially now that running open-weight models on local hardware has moved beyond experimental phases and into regular practice.

Just last month, David Kaplan, AI Security Research Lead at Origin, conducted a similar experiment. He created a compromised model designed to steal data. If a pharmaceutical company were to use this model in a drug development context, it could silently exfiltrate data via calls to the send_email tool without the user realizing it.

Kaplan critiqued the “lethal trifecta” AI threat model, popularized by developer Simon Willison, which outlines three elements: private data, untrusted input, and an external output mechanism. Kaplan argued that this framing underestimates the risks. “In this case, the three elements are not even necessary. All that’s needed is one outbound tool and model weights that quietly decide to use it. The ‘untrusted input’ doesn’t come from a web page—it was embedded in the weights from the start.”

When the Assumption of Trust Breaks Down

Open-weight models have been touted as “transparent,” but this experiment challenges that assumption. Even if weights are publicly available, there is no means to verify what is happening internally. With traditional software, disassembling binaries and analyzing control flows could uncover malicious behaviors. In contrast, neural network weights are merely numerical matrices, making it virtually impossible to decipher intended behaviors.

This issue is particularly serious in today’s world, where AI models are circulated as part of supply chains. Providers of these models may claim they are secure, but users have no way to verify these claims. Paxton-Fear’s experiment shows that poisoning a model is achievable by virtually anyone with under $100 and basic tools, meaning the barriers for malicious attackers to adopt similar methods are extremely low.

For instance, imagine a company downloading an open-weight model from a repository like Hugging Face to use in internal code generation or data analysis. If that model had a cleverly embedded backdoor, it could generate code with vulnerabilities allowing for remote code execution or send sensitive data to external entities. Unlike traditional supply chain attacks, the behavior of such a model is extremely difficult to verify beforehand, making detection nearly impossible.

Industry Responses and Challenges

In the field of AI security, several methods have been proposed for verifying models, such as red-teaming (adversarial testing), model interpretability techniques, and tamper detection through cryptographic hashes of weights. However, this research suggests that these measures remain inadequate. Poisoning through fine-tuning can be hard to detect because it only involves small differences from the original weights, and the current lack of mechanisms to prevent training data contamination makes defense challenging.

Moreover, even officially released weights from model providers are not immune to tampering risks somewhere along the supply chain. While repositories like Hugging Face conduct scans, instances of malicious models slipping through have been reported. The study’s finding that larger models are easier to poison is particularly alarming. With more parameters, there are more opportunities to embed subtle changes, making detection even harder.

Although closed models like OpenAI’s recently announced GPT-5.6 models (Sol, Terra, and Luna) might seem more secure due to their API-only access, they are not immune to supply chain risks, such as weight tampering. Open-weight models, however, are inherently more vulnerable because they can be freely downloaded, modified, and redistributed.

Editorial Opinion

In the short term, this research serves as a wake-up call for security professionals in organizations adopting AI models. In environments where model fine-tuning and repurposing are common, evaluating supply chain risks becomes even more critical. Organizations, especially those using models for code generation or internal tools, must strictly manage the provenance and modification history of their models. Over the next 3–6 months, we can expect accelerated investment in automated tools for model verification and in provable methods to ensure the integrity of model weights.

In the long term, the ecosystem for open-weight models stands at a crossroads. As this study demonstrates, the current distribution model for open-weight models, which allows unverifiable behavior, cannot ensure trustworthiness. Over the next 1–3 years, the industry may standardize cryptographic signatures and traceability mechanisms for model weights. However, as current technologies lack the ability to fully verify internal model behaviors, the AI supply chain will continue to face inherent risks until this fundamental problem is resolved.

As a point of reflection, we urge model providers to reconsider their claims of “transparency,” as users currently lack the tools to independently verify the behavior of these models.

References

• “Researcher poisons open-weight AI model for under $100”, by Thomas Claburn — The Register, 2026-07-16T20:25:00.000Z (ARR)
• Source URL: https://www.theregister.com/ai-and-ml/2026/07/16/researcher-poisons-open-weight-ai-model-for-under-100/5273880