Ordinary security protects the software around a model. Adversarial machine learning is about attacking the model itself, and several of these attacks exploit properties of how learning works rather than mistakes anybody made.
Evasion, or adversarial examples
An input changed slightly, often imperceptibly, so the model gets it wrong while a person sees nothing unusual. A few pixels alter a classification. A sticker on a sign changes what a vision system reads.
The unsettling part is that this is not a bug in a particular model. It follows from high dimensional decision boundaries, and it has resisted a general fix for over a decade. Adversarial training, which trains on attacked examples, helps against the attacks it saw and costs accuracy on ordinary input.
Attacks also transfer. One built against a model you have often works against a different model trained for the same task, so keeping weights private protects less than it appears to.
For language models the same family arrives as prompt injection and jailbreaking. The medium is text rather than pixels and the shape is identical, which is an input crafted to move the model somewhere its designers did not intend.
Poisoning
Corrupting what the model learns from. An attacker who can influence training data, or a retrieval index, can plant behaviour that surfaces later.
Backdoors are the sharp version. The model behaves normally except when a trigger appears, at which point it does something chosen by whoever planted it. It passes evaluation, because evaluation does not contain the trigger.
This is the attack that grows as systems pull in more outside content. Anything scraped, crowdsourced, or accepted from users is a surface, and a retrieval index built from documents anybody can edit is poisonable without touching the model at all.
Extraction
Rebuilding a model by querying it. Enough input and output pairs allow training a copy that approximates it, which matters commercially where the model is the product.
Rate limiting and monitoring for systematic querying are the practical defences, and neither is complete.
Membership inference
Determining whether a particular record was in the training data. It sounds minor and is a privacy breach on its own, because membership can be the sensitive fact. Knowing somebody's record was in a dataset of patients with a condition discloses the condition.
Model theft and supply chain
The ordinary security problems, which are also the most likely. Weights left in accessible storage, a compromised dependency in the training pipeline, a model downloaded from a public hub with no verification of what is inside it.
Less interesting than adversarial examples and considerably more likely to be how a real incident starts.
What this means in practice
Two things worth carrying.
Treat model input as untrusted, always. Whether it arrives as an image, a document or a prompt, it may be constructed to move the model.
Assume defences are partial. None of these attacks has a complete fix, so the security case cannot rest on the model resisting them. It rests on limiting what a manipulated model is permitted to do, which is why least privilege and oversight do the work that robustness cannot.