Most of securing an AI system is ordinary security applied carefully. A smaller part is specific to there being a model in the path, and that part is where teams are least practised.
The shared responsibility model
Every major cloud publishes a version of the same idea. The provider secures the cloud itself and you secure what you put in it, with the line between them moving depending on which service you are using.
With a fully managed model service, Amazon Bedrock, Azure OpenAI or Google's Agent Platform, the provider runs the model and the infrastructure, and you own identity, access policy, encryption keys, what you send, and what your application does with what comes back. With a model you host yourself, patching and network configuration become yours as well.
What matters is being able to place a given responsibility on the correct side.
Identity and access
IAM roles, policies and permissions are the primary control. A model endpoint is a resource like any other, and least privilege applies to who can call it, who can change it, and who can read the data it was trained on.
The training data usually carries more risk than the model. It is the largest collection of sensitive material in the project, and it is frequently the least protected, sitting in a bucket somebody opened during development.
Encryption
At rest covers training data, model artefacts, logs and any store of embeddings. Embeddings deserve particular attention, because they are derived from source documents and are often treated as harmless numbers when they can be partially inverted.
In transit covers every call. Where a review requires that traffic never touch the public internet, every platform has a private connectivity product for it, AWS PrivateLink, Azure Private Link and Google Private Service Connect. Encryption protects the content of a request and does not change the path it takes, which is the distinction that requirement turns on.
Data protection services
Sensitive data discovery scans storage and classifies what it finds, which is how you learn that personal information reached a training set. Amazon Macie is the AWS service, Google has Sensitive Data Protection, formerly Cloud DLP, and Azure has Purview.
The distinction worth holding is between locating sensitive data and protecting it. Discovery answers the first question, and encryption and access policy answer the second.
Data lineage and cataloguing record where data came from and how it moved. This is a security control as well as a governance one, since you cannot honour a deletion request or answer a breach question without knowing what a dataset contains and what was trained on it.
Secure data engineering
The syllabus names four practices. Assessing data quality, since bad data produces bad decisions at scale. Privacy enhancing technologies, such as anonymisation, tokenisation and masking before data reaches training. Access control, applied to the data as tightly as to the model. Data integrity, so nobody can alter a training set or an index without it being noticed, which is the defence against poisoning.
The AI specific surfaces
Prompt injection is untrusted text that overrides your instructions. It matters most when the model can act, since an agent with tools turns a text attack into a real one. The controls are filtering input and output, giving tools the narrowest permission that works, and treating model output as untrusted input to whatever consumes it.
Data poisoning corrupts training data or a retrieval index so the system learns or repeats something planted.
Model extraction rebuilds a model by querying it enough times, which is what rate limiting on an endpoint is partly for.
Sensitive disclosure is the model repeating something from training data or from context. Anything placed in a prompt can come back out.
Managed filtering is the control that helps, applying content filters, denied topics and word filters to both directions of the exchange. Guardrails for Amazon Bedrock, Azure AI Content Safety and Google's safety settings and Model Armor all do this, and none of them removes the need to treat model output as untrusted input downstream.
The ordinary practices still apply. Application security, vulnerability management, infrastructure protection and threat detection. The tooling is whatever you already use for any other workload, GuardDuty, Inspector and CloudTrail on AWS, Defender for Cloud on Azure, Security Command Center on Google.
Practise this
You need one sheet of paper and fifteen minutes. Use no diagramming tool, because what you can produce without one is the whole point of the exercise.
Draw your feature from memory before opening any code. Mark two things separately, every arrow carrying content you did not write, and every box that can change something outside your own process. Then open the repository and correct the drawing against what is actually wired up.
Draw it from memory first, then correct it.
[ user ] --> [ your app ] --> [ model ]
| |
[ retrieval ] [ tools ]
| |
[ documents ] [ effects ]
Mark every place untrusted content enters:
user message, uploaded file, retrieved chunk, tool result,
fetched web page, email or ticket body, another agent's output
Mark every place the system can act:
writes a record, sends a message, calls a paid API, moves money,
changes a permission, deletes something
For each action, answer: whose credential, and can it be undone?
Look for the arrows missing from the first drawing, which are almost always the tool result coming back into context and the retrieved chunk, since both feel like your own data and neither one is. The distance between the drawing from memory and the corrected one is a fair measure of how well the boundary is understood by the people maintaining it.
Nobody defends a boundary they cannot draw, and most teams find out here that they cannot draw theirs.