An LLM feature looks free until the monthly invoice arrives. This post covers what drives the cost per user and where startups lose margin most often.
What one request costs
Start with the basic formula: request cost = input token price × input tokens + output token price × output tokens. Every provider charges several times more for output than for input, so the length of the answer moves the bill more than the length of the prompt.
If the feature writes long text (emails, product descriptions, commented code), most of the spend is generation, not the context you send in.
One user, many requests
The usual early mistake is to price a single API call and multiply it by the number of users. A real user makes several calls per session, and the arithmetic changes.
A conversation grows with each turn. The fifth question in a chat drags the whole history into the context, so you pay for everything accumulated so far, not for one short message. An agent that plans its own steps can fire a dozen calls for a single user task.
That makes the session, or the whole task, the right unit. Measure how many calls one user action takes on average and multiply the per-call cost by that number, not by one.
What breaks the margin
Three things break unit economics most often.
Heavy users. Usage is almost always skewed: a small share of accounts produces most of the tokens. With a flat feature price those accounts lose money, and the rest of the user base covers them through the shared subscription price.
Retries. The model got it wrong, the app asked again, then again. Every attempt is billed in full, while the business metric sees only the final success and none of the cost behind it.
Growing context in long sessions. A support bot that carries the full conversation pays for that history with every new message. Without trimming or summarizing, one long session can cost several times as much as a short one.
How to work out the cost of a feature
Collect three numbers over a representative period: average token spend per user action, average number of actions per user per month, and the share of failed calls that had to be repeated.
Then the formula: monthly cost per user = number of actions × average cost per action × retry adjustment. The adjustment usually sits between 1.1 and 1.3 when the feature retries automatically.
Read percentiles, not only the mean. The median user may cost pennies while the 95th percentile costs several times more. A price built on the average often hides how much money the busiest part of your audience loses you.
How to set the price and stay positive
A flat subscription works while usage varies little between users. When the spread is wide, build a limit into the plan: a fixed number of requests per month at a fixed price, then an add-on or an upgrade beyond it.
A second move is to price against a buffer instead of against today's model cost, so a future price increase or a switch to a stronger model does not wipe you out. A margin calculated right up against current API rates disappears the moment a provider changes them.
How to catch a cost spike early
A single calculation at launch is not enough. You need a standing view of where the spend comes from: which feature, which user, which model.
codexkey tracks usage per API key. Issue separate keys per feature or per customer segment and you can see which one is eating the budget. Sub-organizations with their own limits help once the team grows and there is more than one product: each gets its own budget and its own warning threshold, not one shared pool that nobody personally owns.
Short version
The cost of an LLM feature is not the price of one API call. Price per session, not per request, check percentiles next to the average, and keep a buffer for the model getting more expensive. Spend broken down by key and by feature replaces guessing with numbers.