AI Model Deprecation: What Happens and How to Prepare

AI model deprecation is when a provider announces that a model you build on will stop working on a set date. If your product calls that model, you have a deadline you did not choose. The related post on platform risk shows what happens to a business when the model under it changes, costs more or disappears, and it helps anyone who ships a product on a model API. It names three ways the ground can move: the model changes, costs more or disappears. A deprecation is the third.

Age of AI Entrepreneur teaches how modern AI works and how to build, launch and grow a business with it, in short hands-on lessons. This post is narrower. It covers what a deprecation means for you and what to do before the notice arrives.

What AI model deprecation means

Deprecation is a process, not a single moment. A provider marks a model as on its way out, sets a date after which requests to it fail, and usually points you to a replacement. Between the notice and the shutdown, the model often keeps working as normal.

Two words get mixed up, so keep them apart:

• Deprecated means the model is still there but scheduled to go.
• Retired means it is gone and calls to it return errors.

The practical point is that the model name in your code is a dependency, like any other. It can be removed.

Why providers retire models

You do not need inside knowledge to see the pattern. Running old models costs a provider effort and hardware, and newer models usually replace them. Providers also change what they offer as the field moves. A model is a product with a life span, not a fixed fact of the world.

This ties to the history of the field. If you want the background on how fast the underlying technology has moved, the course The Road to the Age of AI walks through it (sign in to open the course). Fast change at the top means turnover underneath you.

What breaks when your model is retired

The obvious break is an error. Your app calls a model name that no longer exists and requests fail. That is the easy case because you see it at once.

The harder break is quality. When you move to the replacement, the same prompt can give different output. Possible effects:

• Answers change in length, tone or format.
• A strict output format, such as JSON you parse, stops matching.
• Cases that worked before now fail, and cases that failed now work.
• Cost per request changes, which moves your margins.

That last point matters. The lesson Model APIs, Tokens and What a Request Costs, in The AI Product Stack, covers how a request is priced. A new model may use tokens differently, so your unit economics can shift even if everything else looks fine. The course Business Basics for Builders covers revenue, costs and unit economics if you want to check what a cost change does to your margin. Sign in to open it.

How to read a deprecation notice and timeline

Notices differ by provider, so read the one you get. Pull out four things and write them down:

1. The exact model names affected. Check every place you use them, including scripts and automations, not just your main app.
2. The shutdown date. Put it in your calendar, then add an earlier internal date.
3. The suggested replacement. It is a suggestion, not a promise that it behaves the same.
4. Any change in pricing or limits that comes with the replacement.

Work backward from the shutdown date. If you need two weeks to test and a week to roll out, your real deadline is well before the provider's. Do not wait for the last month.

Build a migration checklist before you need it

Write the checklist now, while nothing is on fire. Here is a version you can copy and adjust:

• List every place a model name appears: code, config, no-code tools, scheduled jobs.
• Record, for each, what the model is used for and how bad a failure would be.
• Save your current prompts and settings in version control.
• Keep a set of real example inputs with the outputs you consider good.
• Name one person who owns the migration, even if that person is you.
• Decide how you will roll out: all users at once, or a small share first.
• Decide how you will roll back if the new model does worse.

The third lesson of Distribution and Moats, Building on Someone Else's Model, is about exactly this kind of dependence. Use it to think about which parts of your product depend on one provider and which do not.

Use an eval set to test the replacement model

An eval set is a collection of inputs with a way to judge the outputs. It turns "does the new model feel okay?" into something you can measure. This is the single most useful thing to have on migration day.

A small worked example. Say your product turns customer emails into a short summary and a priority label. Build a file of 30 real emails, with the summary and label you would accept for each. When the replacement model arrives:

1. Run all 30 emails through the old model and the new one with the same prompt.
2. Check the label against your expected label. That part is easy to score.
3. Read the summaries side by side and mark each as better, same or worse.
4. Look at the failures. If they cluster, for example on long emails, change the prompt and run again.

Include your awkward cases, not just the easy ones. The course Evals: Shipping AI That Works has lessons on why a product needs evals and on building an eval set and judging outputs. Its lesson on running AI in production is relevant once you roll out.

Design your product so swapping models is easy

You cannot stop a provider retiring a model. You can make the swap cheap. A few design habits help:

• Keep the model name in one config value, not scattered through the code.
• Put all model calls behind one function in your code, so the rest of the app never talks to a provider directly.
• Keep prompts in files you can version, not buried in strings.
• Validate outputs. If you expect JSON, check it and handle bad results instead of trusting them.
• Log which model produced each output, so you can find problems after a switch.

Choosing between model types is part of this too. The lesson Choosing Models: Big, Small, Open, Closed in the AI Product Stack course looks at those trade-offs. If your core value is a thin layer over one model, a swap hurts more. The lesson Thin Wrapper or Real Product? in Finding AI Opportunities helps you judge where you stand.

If you are not technical, the post AI explained for non-technical founders is a good place to get the vocabulary so you can ask your developer the right questions.

Frequently asked questions

How much notice do AI providers give before retiring a model?

It varies by provider and by model, so read the notice you receive and check the provider's own documentation. Plan as if the notice could be short, and keep your migration checklist ready.

Will my prompts work the same on a newer model?

Not always. A newer model can follow instructions differently, change its format or handle edge cases in a new way. Run your eval set before you switch, and adjust the prompt if scores drop.

Should I pin a specific model version?

Pinning a version keeps behavior steady while that version lasts, which makes testing easier. It does not stop the version being retired, so you still need a migration plan.

Do I need a backup provider?

It depends on how much a failure would cost you. At minimum, structure your code so another model can be swapped in, and test one alternative against your eval set so you know it is an option.

Get started

Start with two small jobs this week: list every place your product names a model, and save 20 to 30 real examples with the outputs you want. Those two things cut most of the pain from a deprecation. For the bigger picture of dependence on a provider, read the post on platform risk.

If you want to learn the skills behind this, Age of AI Entrepreneur has short hands-on lessons on the AI product stack, evals and building on someone else's model. You sign in with your email and an emailed code, and the terms page explains that every lesson is free to anyone with an account.