What Should AI Cost? Benchmarks by Product Shape
The question comes up in every diligence process and it is almost always asked in the least useful form: what percentage of revenue should AI cost?
There is no single answer, because the number depends almost entirely on where the model sits in the product. A company that uses a model to summarize a support ticket once per conversation and a company whose product is a continuous agentic loop over a large document corpus are both AI companies, and their cost structures differ by more than an order of magnitude. Benchmarking one against the other produces a number that is precisely wrong.
What is portable is the method, and the shape of the answer by product archetype.
Four archetypes
Feature-attached AI. The model powers a discrete feature inside a product that would exist without it — summarization, drafting, classification, search ranking. Inference fires on explicit user action, once or a handful of times per session. This is the large majority of companies currently described as AI-powered.
At this shape, AI cost typically lands in the low single digits as a share of revenue, and gross margin is barely disturbed. That is a genuine finding, and it cuts both ways: the economics are safe, and the AI is not the business. A premium justified by AI on a company in this category is usually a premium being paid for a feature.
Workflow-embedded AI. The model is in the path of the core workflow and fires many times per unit of work — document processing, code assistance, agent-assisted operations. Volume scales with customer activity rather than with seat count.
Here AI cost commonly runs in the high single digits to the low twenties as a share of revenue, and it is the single largest component of cost of revenue after hosting. This is the band where the economics are genuinely interesting and where optimization has real EBITDA consequence. It is also the band where the mismatch between usage-scaled cost and seat-scaled pricing does the most damage.
AI-as-the-product. The output the customer buys is model output. There is no meaningful product underneath. Volume is unbounded by design, because more usage is the value proposition.
Cost here frequently sits in the twenties to forties as a share of revenue and can go higher. Companies in this category have usually been forced to confront their economics already, and have either moved to usage-based pricing or are burning capital deliberately. If you find one that has done neither, that is the finding.
Agentic or multi-step systems. The system takes many model calls per task, often with retries, tool use, and reflection loops. One user request can fan out to dozens or hundreds of inference calls.
This is where cost is least predictable and where I have seen the largest surprises. Cost per task can vary by a factor of ten between an easy and a hard case, which means a single large customer with hard cases can move the blended margin materially. Benchmarks are close to useless here. What matters is the distribution, not the mean.
The two numbers that actually matter
Regardless of archetype, the diligence reduces to two figures.
The first is AI cost per unit of the thing you charge for. Not total inference spend, which is uninformative, and not cost per user, which is misleading in any usage-based product. Per contracted unit — per seat if you sell seats, per transaction if you sell transactions, per document if you sell documents. This is the number that lets you compare cost against price at the level the contract operates.
The second is the trajectory of that number. A company whose cost per unit is falling is getting better at this: it is tiering models, caching, shortening context, or renegotiating rates. A company whose cost per unit is flat is not managing it, and a company whose cost per unit is rising has a product that is getting more expensive to deliver as it matures. That last case is more common than it sounds, because feature expansion in AI products usually means more model calls.
Two companies with identical current cost ratios and opposite trajectories are not comparable investments.
Why the absolute number misleads
Three reasons the share-of-revenue figure alone will lead you astray.
Pricing structure dominates. A company at 15 percent AI cost with usage-based pricing and contractual usage caps is in a fundamentally safer position than a company at 8 percent with flat per-seat pricing and unbounded usage. The second company's ratio is better today and worse at every future volume. Ratio without pricing structure is not a benchmark, it is a snapshot.
Growth changes the number in the wrong direction. Inference is variable cost. In a conventional software business, growth improves gross margin because fixed costs amortize. In a usage-heavy AI business, growth holds gross margin flat at best, and degrades it whenever pricing is not usage-linked. The instinct that scale fixes margin is simply wrong here, and it is the most common modeling error I see.
Reported cost may be understated. Human review of model output is a cost of revenue and is frequently booked as operating expense. Training and fine-tuning runs are sometimes capitalized. Provider credits and startup programs expire, and a company on discounted or free credits has a cost structure that has not been tested at list price. Ask what the bill looks like without credits, and ask when they expire.
How to run the benchmark
Four steps, and none of them require the company's cooperation beyond the data room.
First, classify the product into one of the four archetypes above. Do this from the architecture and the request pattern, not from the marketing.
Second, get inference spend by month for at least four quarters, and the corresponding volume of the contracted unit. Compute cost per unit by month. The series matters more than any single point.
Third, decompose the cost. Which model, which call sites, which customers. In nearly every environment I have looked at, a small number of call sites account for most of the spend, and a small number of customers account for most of the volume. That concentration is where both the risk and the optimization live.
Fourth, model the margin at plan volume rather than at current volume, holding pricing at contracted terms. If the growth plan triples usage, triple the variable cost and see what happens to the gross margin line. Then attach a confidence level to the optimization you believe is available, rather than assuming it.
What good looks like
A company managing this well has a cost-per-unit series that trends down, a routing policy that sends easy requests to cheap models, an evaluation set that makes model changes safe, caching on repeated context, and someone accountable for the number by name.
A company managing it badly sends every request to the most capable model available, cannot attribute cost to customers or features, prices per seat while usage grows, and reports AI cost only in aggregate.
The second company is not necessarily a worse investment. The optimization available is real and often large — I have seen tiering alone cut inference cost by 60 to 80 percent. But it is a different investment thesis, with the value creation in the fixing rather than in the buying, and it should be priced that way.
Benchmarking a target's AI cost structure is the core of an AI unit economics review. Related reading: When AI-Powered Quietly Breaks Gross Margin on the mechanism, and Model Tiering on the largest single optimization. Book a free discovery call.