AI and cybersecurity

20.1AI in the service of attack

20.2AI in the service of defense

In context

When AI finds vulnerabilities better than humans (the turning point of 2026)

A threshold was crossed in 2026: the best frontier models became capable of discovering and exploiting software vulnerabilities at the level of the very best human experts, on an unprecedented scale. Anthropic's Mythos model thus uncovered thousands of unknown vulnerabilities ("zero-days") across all the major operating systems and browsers, some of which had gone unnoticed for decades (a 27-year-old flaw in OpenBSD, another 16-year-old one in the FFmpeg video component that no automated test had detected). This is a characteristic double-edged sword: the same capability that makes a model dangerous in the wrong hands makes it precious for patching vulnerabilities before they are exploited. Hence, in April 2026, the Glasswing project, a defensive coalition bringing together Anthropic and major players in tech and finance (Amazon, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA, Palo Alto Networks, etc.) to put these capabilities in the service of security, including for maintainers of open source software; other labs are developing their own tools in parallel (for example Big Sleep and CodeMender at Google). The flip side appeared in June 2026: invoking export controls and national security, the U.S. government suspended, overnight, access to the Fable and Mythos models for any foreign national, out of fear that their cyber capabilities might be misused. The episode fueled a heated debate: for the government, it was a matter of national security; for Anthropic, the measure was disproportionate and rested on a minor workaround that other public models also exhibited; for some observers, a product presented as a near-"weapon" ends up being treated as such by the state. It is one of the first cases where an export control targets an AI model, and no longer only chips (Chapters 22 and 25).

20.3The security of AI systems themselves

In plain terms

Here is a new and crucial dimension: AI systems are themselves an attack surface. Securing AI has become a discipline in its own right, because models present unprecedented vulnerabilities:

Prompt injection: slipping, into a web page or a document the AI will read, hidden orders that hijack an agent (Chapter 6), for example to make it exfiltrate data. It is one of the most worrying vulnerabilities of agents.
Jailbreak: crafting requests that push a model to break its own rules.
Data poisoning: corrupting training data to introduce malicious behaviors or backdoors.
Model theft: extracting, through repeated querying, the capabilities or parameters of a proprietary model.
The supply chain: downloading a compromised open source model or component (booby-trapped weights, malicious dependency), or deploying unaudited "vibe-coded" code (Section 6.7) that carries vulnerabilities. The Moltbook case (Chapter 6) provided an illustration of this.
"Shadow AI": the use, by employees, of unapproved AI tools into which sensitive company data may leak without their knowledge.

In context

Two subtler attacks, and the frameworks to defend against them

To the vulnerabilities above are added two attacks that target the model itself in a more insidious way. Adversarial examples consist of modifying an input in a way that is imperceptible to a human but that fools the model: a few altered pixels and a "stop" sign is read as a speed limit, or a sticker makes an object vanish from the eyes of a vision system. Membership inference, for its part, seeks to guess whether a specific piece of data (a person's medical record, for example) was part of the training data, which poses a privacy problem. Faced with this zoology of AI-specific threats, reference frameworks have taken shape to catalog them and guide defenders, notably the MITRE ATLAS matrix (the AI equivalent of the classic attack-tactic databases) and a "top 10" of the risks of applications based on large models published by OWASP. Knowing the map of attacks is the prerequisite for any methodical defense.

In context

Distillation, or how to copy an AI without touching its code

The model theft mentioned above has a particular form, which became a burning topic in 2026: distillation. The principle, otherwise common and legitimate (Chapter 4), consists of training a model to imitate the responses of another, more capable one. Applied without authorization to a competitor's closed model, it amounts to capturing its know-how through the public interface alone: you bombard the API with questions, record the outputs, and use them to train a rival system at lower cost, without paying for the fundamental research. In June 2026, Anthropic thus accused the Chinese lab Qwen (Alibaba), in a letter to the U.S. Senate, of having carried out the largest known operation of the kind: roughly 28 million exchanges via tens of thousands of fake accounts to bypass the guardrails; similar accusations had already targeted DeepSeek, Moonshot, and MiniMax, and OpenAI had made comparable ones earlier. The problem is formidable: the product of an AI is its responses, and one can hardly sell them while preventing people from reading them. The countermeasures (watermarking of outputs, scrambling of reasoning traces, behavioral detection) remain imperfect. A safety risk is grafted onto this, finally: a model cloned through distillation inherits the capabilities of the original without necessarily its guardrails (Chapter 24).

20.4A permanent offense-defense race

Diagram20.1. The three faces of AI in cybersecurity. AI is at once an offensive weapon, a defensive shield, and a new system to protect. The three advance together, in a permanent race.

In context

Toward autonomous cyberattack agents

The great shift under way is the move from tools that assist a human hacker to agents (Chapter 6) capable of chaining together the steps of an attack on their own: reconnaissance of a target, search for a vulnerability, exploitation, then progression through the system, with minimal human intervention. The 2026 turning point on vulnerability discovery (Section 20.3) is the harbinger of this: once a model knows how to find vulnerabilities at scale, the next step is to have it orchestrate the complete chain. Labs and security companies, moreover, reported, in 2025–2026, real malicious operations where a large part of the work was delegated to an agent. The same autonomy obviously serves defense (agents that monitor, detect, and respond continuously, faster than a human team), which revives the race of the previous section, but at machine pace. This is also why these capabilities are at the heart of safety evaluations (Chapter 24) and of the question of autonomy, one of the thresholds that labs watch most closely. As elsewhere, the course stays at the level of risk, with no technical detail whatsoever.

On the horizon, the prospect of increasingly autonomous cyberattacks (and cyberdefenses) raises issues that go beyond technology alone and connect to defense (Chapter 22), safety (Chapter 24), and governance (Chapter 25). Cybersecurity thus becomes one of the arenas where the mastery of AI will concretely be decided.

Key takeaways (Chapter 20)

AI lowers the cost of cyberattacks (phishing at scale, deepfake fraud, assistance with malicious code, offensive agents): a democratization of offensive capability.
It is also a powerful defense (anomaly detection, automated response, threat analysis), but an asymmetry could favor the offense in the short term.
AI systems themselves are vulnerable: prompt injection, jailbreak, data poisoning, model theft. Securing AI is a new discipline.
It is a permanent attack-defense race, which connects to defense (Ch. 22), safety (Ch. 24), and governance (Ch. 25).

From technical threats to the frameworks meant to govern them: Chapter 21 takes up law and society, and Chapter 22, defense.

20.1AI in the service of attack#

20.2AI in the service of defense#

20.3The security of AI systems themselves#

20.4A permanent offense-defense race#

Key takeaways (Chapter 20)

20.1AI in the service of attack

20.2AI in the service of defense

20.3The security of AI systems themselves

20.4A permanent offense-defense race