What does adding AI to your app actually cost?

Where does the cost actually sit?

There are three lines, and their order surprises most teams. First, development: designing the feature, writing the prompts, building an evaluation set, building the interface. Second, model usage: the token fee paid per request. Third, maintenance: models change, prompts go stale, user behaviour drifts.

Most teams focus on the second line because it is the only one with a published price list. Yet in a small or mid-sized product the model bill is usually the smallest line; the real money goes to development and — after the first year — maintenance.

The unit is tokens, not users

Model pricing runs on tokens: the text entering the model is priced as input, the text it produces as output, and output always costs more. The practical consequence is that two products with identical user counts can differ several-fold in cost.

What makes the difference is the flow. A product that sends forty pages of context with every request consumes tens of times more tokens than one answering a short question with a short answer — with the same number of users. That is why “how many users” is not enough to estimate cost, and “what is sent on every interaction” is.

Three mechanisms that lower the cost

• **Prompt caching.** The fixed context repeated on every request — system instructions, product rules, a document — is written once and read on subsequent requests. On Anthropic's pricing page a cache read is roughly a tenth of the normal input rate. In any product with repeated context this is the single biggest lever.
• **Batch processing.** Work that does not need an instant answer — overnight classification, bulk summarisation, data enrichment — runs at half the standard price through the batch endpoint.
• **Splitting across models.** Give mechanical steps like classification, routing and formatting to a small model, and the step that needs reasoning to a strong one. An architecture that does everything with one model pays the price of the most expensive step at every step.

Applied together, the difference is a multiple rather than a percentage. But all three are architectural decisions, not settings added after the feature is written. That is why cost optimisation belongs in the design phase.

Three mistakes that blow up the bill

• **Embedding the API key in the app.** A key embedded in a mobile app can be extracted, and the moment it is, someone else determines your bill. Model calls should always pass through your server; the client should only ever see your endpoint.
• **No per-user limit.** If there is no definition of how many requests a user can make per month, a single automated script can spend the monthly budget in a day. The limit is part of the product, not a security afterthought.
• **Letting context grow forever.** In a flow that accumulates chat history without trimming, every new message costs more than the last; cost rises cumulatively rather than linearly as the conversation lengthens.

Can it be fixed-priced?

Development yes, usage no — and making that split up front protects both sides. Development scope is definable work: the feature, the interface, the evaluation set and delivery. Model usage depends on user behaviour; pricing it as fixed means burying the risk in the price and having you overpay for it.

The structure that works: development at a fixed price, usage transparent and metered directly with the provider. On top, a monthly ceiling defined up front and an alert that fires as the ceiling approaches. Cost is then neither a surprise nor a guess.