The AI supply chain

The AI supply chain is the chain of artefacts a team takes on when it ships an AI feature, most of which it did not make and cannot inspect. The idea comes from software supply chain security, which became a board level concern after a run of compromised build tools and libraries around 2020, and it reached AI when researchers began finding tampered models on the public hubs in 2023 and 2024. The supply chain sits fourth on the OWASP 2026 list for LLM applications, above most of the failures teams spend their attention on, because a dependency that has been tampered with or has quietly changed defeats every control built on top of it.

The sections below say what a feature is actually depending on, where the chain goes wrong, which controls answer it, and end with a fifteen minute inventory.

What a feature is actually depending on

A single feature usually rests on a base model the team did not train and whose training data it cannot see, weights downloaded from a hub, adapters or fine tunes layered on top, the datasets those were tuned on, an embedding model that decides what retrieval can find, a vector store, an orchestration library with its own transitive dependencies, and prompt templates copied from a repository or a blog post.

Each of those is a dependency in the ordinary software sense, and several are invisible to the tooling the team already runs. A vulnerability scanner reads the lockfile. It does not read the model file, the adapter, the index or the prompt template somebody pasted in during a busy week. The software world's answer to that gap is a bill of materials listing every component, and an AI bill of materials extending the idea to models, datasets and prompts has been under standardisation since 2024.

Where the chain goes wrong

Five failures account for most of the incidents.

Tampered or backdoored weights. Some model file formats execute code on load, which is an ordinary deserialisation problem wearing a new hat, and a 2024 scan of a public hub found around a hundred uploaded models carrying such payloads. A safer format that stores only tensors has existed since 2022 and is now the hub default. The harder version is behavioural, since a fine tune can carry a response that appears only on a trigger phrase and is therefore invisible in any sample of normal output, and a 2024 paper showed such behaviour surviving the safety training meant to remove it.

A card that does not match the artefact. Model cards are authored claims about training data, intended use and evaluation results, and nothing ties them to the bytes downloaded. On a public hub the name is the weakest part of the identity, so the card being read may describe something else entirely. A 2023 demonstration uploaded a modified copy of a popular open model under a name one letter away from the original, with a single false historical fact edited into its weights and the original's card attached, and it was downloaded before the hub took it down.

Licensing that surfaces later. Terms vary on commercial use, redistribution and output, and they stack, since a fine tune carries the base model's licence along with whatever the tuning data imposed. The finding usually arrives from a buyer's legal review, long after the team's own build.

Abandoned dependencies. An orchestration library with two maintainers and a large surface, an embedding model nobody has updated in two years, a vector store project that stopped shipping. The risk is the familiar one and this ecosystem produces it unusually fast.

A hosted model changing underneath the product. This has no clean equivalent in ordinary software. A provider can update the model behind an endpoint name, retire the version the team tested, or change a default safety setting. The code did not change and the system now behaves differently, which is the failure that makes people distrust their own regression results.

The controls that answer it

Four controls answer the five failures, and all four are cheap on the day a dependency is taken.

Versions are pinned. A model version and never an alias, a library version and never a range, and the embedding model version in particular, because changing that one forces a reindex. Anything called latest moves without telling anyone.

Downloads are verified. Where a publisher gives checksums or signatures, they are checked and the verified value is recorded alongside the fact of the check. Formats that execute on load are avoided, and anything uncertain is opened somewhere the team would not mind losing. Signing of model artefacts through the same infrastructure used for software packages has been available since 2024 and is spreading through the hubs.

Dependencies are recorded. A list of every model, adapter, dataset, embedding model, index and library the feature uses, with versions and sources. The value is in being able to answer, on the day a hub pulls a model or a licence changes, whether the team is affected, without somebody reading code to find out.

A version is evaluated, and never a name. An eval result belongs to a specific version of a specific chain, and rerunning it after any change in that chain is what turns the inventory from paperwork into a control.

Provenance is cheap to establish on the day a dependency is taken and close to impossible to reconstruct a year later, which is the argument for doing it while it still feels unnecessary.

Practise this

The exercise needs the feature the team most recently shipped, a text file, Python for the second half, and fifteen minutes.

Write the inventory from memory first, one line per artefact with its version and where it came from, and only then check it against the code. Then take the files the team ships but did not write, meaning prompt templates, adapters and index configuration, and record what they are today.

import hashlib, pathlib

# Point this at whatever the feature loads that the team did not author.
for path in sorted(pathlib.Path("./prompts").rglob("*")):
    if path.is_file():
        digest = hashlib.sha256(path.read_bytes()).hexdigest()
        print(f"{digest[:16]}  {path}")

The interesting output is the row that cannot be completed. Somewhere in that inventory is an artefact whose version nobody pinned, whose origin nobody remembers, or whose licence nobody has read, and the gap between the list written from memory and the list in the code is an honest measure of how well the chain is understood.

It teaches that what a team depends on is a specific version of a specific artefact, and that the question a customer eventually asks can only be answered if somebody wrote the answer down at the time.

Common misconceptions

“We use a major provider, so the supply chain is their problem.”

They are responsible for the artefact they publish. The team's chain also holds adapters, an embedding model, a vector store, an orchestration library and its transitive dependencies, and prompt templates somebody copied from a repository. Most of that arrived without a review.

“The model card describes the model.”

A card describes what its author says, and nothing binds it to the file that was downloaded. Cards get copied along with names, so a fine tune uploaded under a similar name can carry a card it never came from. A checksum against a published value is the check, and reading the card is no substitute.

“Open weights mean the team can use it however it likes.”

Open weights are not one licence. Terms differ on commercial use, on redistribution and sometimes on what may be done with the output, and a fine tune inherits constraints from the base model and from the tuning data. This normally surfaces during a customer review, which is the expensive moment to find out.

Where this is examined
AI Safety Practitioner
Regulatory Requirements and Governance, 15 per cent of the exam.
Related material
Book
AI Engineering, On what depending on somebody else's model commits a team to.
Book
Fundamentals of Data Engineering, On lineage, and knowing where an artefact came from.
Template
Launch checklist, Everything that has to happen from two weeks out to one week after release, grouped by when it falls due, each line with a named owner and a go or no go decision on the day.
Concepts