LLM Token Pricing Explained: What One AI Request Costs

LLM token pricing explained in one line: you are billed for the pieces of text, called tokens, that you send to a model and the tokens it writes back, usually at one rate for input and another for output. A founder who ignores this can launch a product whose every conversation loses money. This post shows how to read a price page, count what one request really costs, and spot where the bill grows.

The AI Product Stack course on Age of AI Entrepreneur teaches this in its first lesson, Model APIs, Tokens and What a Request Costs, and its Launch, Price and Get Paid course takes the next step with a lesson on Pricing AI Products. A reader signs in with an email and an emailed code to open either one.

What a token is and why models count in tokens

A token is a piece of text a model reads or writes as one unit. It can be a whole short word, part of a longer word, or a bit of punctuation. Models work with a fixed vocabulary of such pieces. A well-known 2015 paper on subword units describes the idea: instead of a dictionary entry for every word, rare and unknown words are encoded as sequences of smaller units, with one method based on the byte pair encoding compression algorithm.

For your budget, three things follow. A token is not a word, so a count in words is only a rough guide. The same text can use different numbers of tokens in different models, because each model has its own vocabulary. And text in some languages, or text full of numbers and code, often needs more tokens than plain English prose. Count with your provider's own tool, not by guessing.

LLM token pricing: how input and output are billed

Every request to a model API has two sides, and each side has its own rate.

• Input tokens are everything you send: your system instructions, any documents you pasted in, earlier messages in the chat, and the new question.
• Output tokens are the answer the model writes back.

Price pages usually quote each rate per million tokens. Many providers charge more for output than for input, so check the page of the model you use, and check it again before you set your own prices, because rates change. The formula is simple:

cost of one request = input tokens times the input rate, plus output tokens times the output rate.

A worked example: one support reply

The numbers below are invented to show the arithmetic. Use your own provider's rates and your own token counts. Say you run a small shop and your assistant answers support questions. Use these placeholder rates: 3 dollars per million input tokens and 15 dollars per million output tokens.

1. Input. The system instructions take 800 tokens. You paste in 1,200 tokens of shop policy found by search. The customer's question is 100 tokens. That is 2,100 input tokens.
2. Output. The reply is 300 tokens.
3. Input cost. 2,100 times 3 divided by a million is 0.0063 dollars.
4. Output cost. 300 times 15 divided by a million is 0.0045 dollars.
5. Total. 0.0108 dollars, a little over one cent per reply.

Notice that the short answer costs almost as much as the long input. The output is only 300 tokens, but each one costs five times as much as an input token in this example. Notice too that most of the input is text you chose to send, not text the customer typed. That is where your control sits.

Why a conversation costs more than its turns add up to

A model does not remember a chat on its own. To continue a conversation, your app sends the earlier messages again with each new turn, and you pay for them as input tokens each time. Keep the same shop and assume each turn adds 100 tokens of question and 300 tokens of answer to the history.

• Turn 1 sends 2,100 input tokens.
• Turn 2 sends 2,500, because the first question and answer ride along.
• Turns 3, 4 and 5 send 2,900, 3,300 and 3,700.

Over five turns that is 14,500 input tokens and 1,500 output tokens: 0.0435 plus 0.0225, or 0.066 dollars. Five copies of turn 1 would cost 0.054 dollars, so the real conversation costs about 22 percent more, and the gap widens with every extra turn.

Five ways to lower the bill without hurting quality

• Send less retrieved text. Pass only the few passages that answer the question, not whole documents. The same course covers giving models your data with retrieval.
• Shorten the system instructions. A rule that never changes an answer is a cost you pay every request.
• Cap the output. Ask for a short answer and set a maximum length, so the model does not write 800 tokens when 200 will do.
• Trim or summarize old turns. Keep the last few messages and a short summary of the rest.
• Use a smaller model for easy tasks. Test on your own examples first; our post on how to choose an LLM for your AI product explains how.

Change one thing at a time and re-run the same set of examples, so you can see whether quality held. Our checklist on how to test LLM prompts before release shows a simple way to do it.

From cost per request to your price

Once you know the cost of a typical conversation, multiply it by how many your average customer will have in a month. That gives the variable cost you must cover before you earn anything. Our walkthrough on how to calculate unit economics for an AI product takes the next step, and usage based pricing vs subscription sets out the serious arguments for each way of charging. The Launch, Price and Get Paid course has a lesson called Pricing AI Products for the same step, and a reader signs in to open it.

Frequently asked questions

How many words is a token?

There is no fixed ratio, because it depends on the model's vocabulary and on the text. Count real samples of your own text with your provider's tokenizer.

Why do output tokens often cost more than input tokens?

Many providers set a higher output rate, but the reasons and the ratio differ by provider and model. Read the price page for the model you use.

Do I pay for the instructions I send every time?

Yes, system instructions and any pasted documents count as input tokens on each request. Some providers offer discounts for repeated text; check the price page before you rely on one.

Does a longer chat cost more per turn?

Usually yes, because earlier turns are sent again as input. Trimming or summarizing old turns keeps the cost per turn flatter.

Get started

Open Age of AI Entrepreneur, sign in with your email and an emailed code, and work through the AI Product Stack course. Each lesson takes about nine minutes and lists its sources, and the terms page says every lesson is free to anyone with an account.