LLM supply chain security: vet models before you use them

LLM supply chain security is about everything your AI feature depends on that your team did not build: the packages, the model, any adapters, the data, and the provider you send prompts to. A normal web app already has a long list of outside code. An AI app adds model files you cannot read like code, and terms that decide what happens to your users' data.

OWASP covers this as LLM03:2025, Supply Chain. This guide lists the parts, what can go wrong with each, and a review you can run before you adopt a model.

What LLM supply chain security covers

OWASP explains that traditional software supply chain work focuses on code flaws and dependencies. With AI, the risk also covers third-party pre-trained models and the data behind them, which can be tampered with or poisoned. It adds that open models and cheap fine-tuning methods, such as LoRA adapters, bring new risks, and so do models that run on a user's device.

Make a list of these parts for your own app before you read on. Most teams find at least one they had not thought of.

What can go wrong

From OWASP's list of common risks, these are the ones most product teams will meet:

• Old or vulnerable packages. The same problem as any web app, with more risk when the packages run during model development.
• Tampered models. OWASP notes that a model is a binary black box: reading its files tells you little. A model can hide a bias or a backdoor that public safety checks missed.
• Weak provenance. Provenance means proof of where something came from. OWASP warns that model cards and documentation give no guarantee of a model's origin, and an attacker can take over a publisher's account or create a lookalike.
• Unsafe adapters. A LoRA adapter is a small add-on that changes how a base model behaves. A malicious one can compromise the model it is added to.
• Outdated models. Models that are no longer maintained stop getting fixes.
• Unclear terms. A provider's terms and privacy policy may let it use your app's data for training, which could later expose it.
• Licenses. Model and dataset licenses can limit how you use, share or sell what you build.

OWASP's scenarios show these are not theory. One describes attacks on the PyPI package registry that tricked model developers into installing a compromised PyTorch dependency. Another describes PoisonGPT, where a model's parameters were changed directly to spread misinformation.

Packages and frameworks

For code, OWASP's LLM entry points back to A06:2021, Vulnerable and Outdated Components. That entry says you are likely exposed if you do not know the versions of every component you use, including the dependencies of your dependencies, or if you do not scan them regularly.

Its advice fits any team:

1. Remove dependencies and features you do not use.
2. Keep a current list of every component and its version. OWASP's LLM entry calls this a Software Bill of Materials (SBOM) and suggests starting with OWASP CycloneDX.
3. Watch public vulnerability lists for the components you use, and patch on a plan.
4. Get components only from official sources over secure links, and prefer signed packages.
5. Watch for libraries that are no longer maintained.

Apply the same rules to notebooks and training environments, not only production. OWASP says to apply these controls wherever development has access to sensitive data.

Models, adapters and where they come from

Because you cannot inspect a model the way you read code, OWASP leans on origin and testing:

• Use models only from sources you can verify.
• Check integrity with signatures and file hashes, so you know the file you run is the file the publisher released.
• Treat every adapter or merged model as a new model, with the same checks.
• Run your own red teaming and evaluations on the use case you plan. OWASP warns that published benchmarks are not enough, since a model can be fine-tuned to pass them while hiding targeted triggers.
• Have a patching policy, and rely on maintained versions of the model and its API.

A model that can call tools raises the stakes, because a hidden trigger then reaches real actions. Our guide to excessive agency shows how to limit what it can touch.

Terms, licenses and your data

OWASP's first mitigation is to vet data sources and suppliers, including their terms and privacy policies, and to review them again over time in case they change. For a hosted model provider, read three things before you send real user data: whether your inputs can be used for training, how long they are kept, and who can see them.

For licenses, OWASP recommends keeping an inventory of every license involved, for software, tools and datasets, and auditing it regularly.

Worked example: vet a model before you use it

Say your team wants to swap your hosted model for an open model you will run yourselves. Before it reaches users:

1. Write down the model's publisher and the exact file or version. Check that the account is the real publisher, not a lookalike.
2. Compare the file hash or signature with what the publisher lists, and save it so a later swap is visible.
3. Note every adapter or merged model involved, and check each one the same way.
4. Read the license for the model and its datasets, and confirm your use is allowed.
5. Add the model, its runtime and every new package to your SBOM, and run your dependency scan.
6. Run your own test set on your real use case, including questions designed to make it misbehave. Compare the results with your current model.
7. Decide who watches for updates and how fast you will patch.

If any step has no answer, that gap is your first finding. Our threat modeling plan for a small team is a good way to decide which gaps matter most.

Frequently asked questions

Does this apply if I only call a hosted model API?

Yes. Your packages, the provider's terms and the model version you rely on are still part of your supply chain.

What is an SBOM?

A Software Bill of Materials: a list of every component in your app and its version. OWASP recommends one for AI apps and suggests OWASP CycloneDX as a place to start.

Are model cards enough to trust a model?

No. OWASP notes that model cards and documentation give no guarantee of where a model came from.

Is a hosted model or an open model safer?

Each moves the risk to a different place. With a hosted model you depend on the provider's terms, versions and security; with an open model you take on checking the files, adapters and patches yourself, so run the checks above either way.

Get started

Choosing a model, a provider or a vendor? whitehatstoic's independent reviews give a second opinion before you commit: a plain written verdict on a project, codebase or vendor, with a clear recommendation. Already built? Our cybersecurity and AI safety testing covers web apps and AI systems, with a written report, fixes and a retest after fixes. Every engagement is scoped after a short call.

Book a meeting with whitehatstoic: tell us the product, the deadline and your biggest worry, and we reply with a plan and a price.