Emergence of "HalluSquatting" Attacks Exploiting AI Hallucinations

Agent-based AI systems are now facing a new security threat. A research team from Tel Aviv University, Technion, and Intuit has introduced a novel attack method called “HalluSquatting” (Hostile Hallucination Squatting). This technique leverages the hallucination tendencies of large language models (LLMs) to trick agents into executing malicious code. According to a report by Tom’s Hardware, this attack targets “a fundamental weakness inherent in every available model.”

Basic Principle of the Attack

The mechanism of HalluSquatting is strikingly simple, yet its impact can be severe. When an LLM encounters unknown terms or newly emerged software repositories not covered in its training data, it hallucinates plausible but incorrect “answers” with high confidence. Research has shown that agents hallucinate malicious code repositories with up to an 85% probability.

What distinguishes this attack from traditional typo-squatting (exploiting typos in domain names) is its direct exploitation of hallucinations. LLMs tend to generate responses using predictable naming patterns (e.g., “owner/repository” formats or “toolname/toolname” GitHub URLs) in uncertain situations.

Execution Steps of the Attack

First, attackers identify popular applications, code repositories, libraries, or bot skills that have gained traction in recent months or years. For example, consider a newly launched GitHub repository named “OriginalOwner/WindowsTelemetryOff.” Since this repository is not part of the model’s training data, variations like “SuperHacker/WindowsTelemetryOff” or “WindowsTelemetryOff/WindowsTelemetryOff,” created by attackers, are treated as equally valid candidates by the model. Typos like “WindowsTelemetryOf” or “WindowTelemetryOff” are also exploited.

Attackers then create malicious repositories with these generated names. When a code agent like Claude is instructed to “run the WindowsTelemetryOff script,” the model might even perform a web search, hallucinate the fake repository as the legitimate one, and download the attacker’s version to execute it.

Chain Reaction and Botnet Formation

Once the attacker’s code is executed on a user’s machine, the damage can escalate rapidly. A direct consequence could be the creation of a reverse shell, enabling attackers to remotely control the user’s command line. After gaining access to the user’s account, attackers can steal data or passwords, install software, deploy cryptocurrency miners, or even exploit the AI agent itself to carry out further attacks.

The research team warns that a single piece of HalluSquatted software could attract tens of thousands of bots in no time. Sophisticated attackers could even embed the original code in the malicious version to avoid raising suspicions, allowing it to operate unnoticed. This could result in the creation of large-scale botnets, enabling attackers to control AI agents without direct involvement.

The Inherent Vulnerability of Agent-Based AI

Security researchers have long cautioned against granting AI agents user-level permissions. The risks of allowing non-deterministic programs with inconsistent outputs to execute code or manipulate files are evident. HalluSquatting demonstrates how these risks can be realized, potentially impacting the entire software supply chain.

Editorial Opinion

HalluSquatting stands out from traditional exploits because it weaponizes the very outputs generated by AI agents’ overconfidence in their hallucinations. In the short term, developers utilizing code agents must urgently implement mechanisms to prevent automatic trust in URLs or repository names generated by AI tools. Examples include maintaining a whitelist of verified repositories and requiring user confirmation before execution.

In the long term, it will be necessary to redesign systems to address software introduced after the cutoff date of training data. Developers must create mechanisms that enable models to explicitly state “I don’t know” for unknown queries and reliably cross-check responses with external knowledge bases.

This attack highlights a broader issue of “trust design” in AI agents. Developers should not only focus on improving model accuracy but also prioritize establishing architectures that ensure the safe execution of tools. Discussions on these topics must intensify to address the vulnerabilities of current AI systems effectively.

References

• Tom’s Hardware — Published on 2026-07-09