Token Economics

Executive Summary

Every LLM provider publishes a pricing table: $X per million input tokens, $Y per million output tokens. The napkin math is seductive. 10,000 requests/day × 2,000 input tokens × $2.50/M = $18.25/day on GPT-4o. Annualized: $6,660. The CFO approves it. Three months later the bill is $54/day — $19,710/year — and nobody can explain the gap.

The gap isn't a billing error. It's five structural leaks between the pricing page and your credit card: workload ratio asymmetry (output tokens cost 3–4× more than input tokens, and your workload determines the ratio), tokenizer variance (the same text is 5–15% more tokens on one provider's tokenizer than another's), unused prompt caching (Anthropic gives 90% off cached input, OpenAI 50% — and most teams never configure it), batch processing discounts (50% off for async workloads — sitting on the table for any non-realtime use case), and rate-limit retry overhead (failed requests get retried, consuming tokens twice).

This briefing deconstructs each leak with real pricing data from OpenAI, Anthropic, Google Gemini, DeepSeek, Mistral, and Meta/Groq. We run the breakeven math for self-hosted vs API across model scales. And we provide a practical decision framework that turns the five leaks into five cost levers — things you can configure today to cut your bill by 40–65% before changing a single line of application code.

Leak #1: Workload Ratio Asymmetry — Your Use Case Is the Multiplier

LLM pricing pages separate input and output token costs. The ratio between them is determined by your workload — and it's the single largest variable in your final bill. Output tokens cost 3–5× more than input tokens at every major provider. The workload that minimizes output tokens (summarization) is fundamentally cheaper than the workload that maximizes them (code generation), even at the same provider with the same daily request count.

The Workload Cost Spectrum

Consider five canonical workloads, each at 10,000 requests/day on GPT-4o ($2.50/M input, $10/M output):

The same 10,000 requests/day on the same GPT-4o model spans from $47,450/year (chat) to $136,875/year (translation) — a 2.9× spread — driven entirely by the input-to-output token ratio. This is before comparing providers. When you layer provider choice on top, the spread widens to 10× or more.

The fix: Measure your actual input-to-output token ratio in production before estimating costs. Most teams guess 1:1 because that's what the pricing page implies. Almost no real workload is 1:1. The LLM API Pricing Calculator includes workload presets for chat, summarization, code generation, RAG, and translation — select the one that matches your use case, then adjust token counts to your measured values.

Leak #2: Tokenizer Variance — 1,000 Tokens Isn't Universal

Every LLM provider uses a different tokenizer. OpenAI uses cl100k_base (tiktoken). Anthropic uses a proprietary BPE tokenizer optimized for conversational English. Google Gemini uses SentencePiece. DeepSeek uses a BPE tokenizer optimized for Chinese+English bilingual text. Mistral uses a SentencePiece derivative. The same 10,000-character document produces a different token count on each — and that difference compounds with every request.

For English text, the variance is typically 5–15%. For multilingual text (especially CJK languages), the variance can reach 15–30%. A Chinese legal document with interspersed English technical terms hits the worst case on several tokenizers simultaneously.

Why this matters: When you compare per-token pricing across providers, you're comparing different units. Provider A at $2.00/M tokens with a 10% less efficient tokenizer is effectively the same price as Provider B at $2.20/M tokens — but only for your specific content type. A provider that's cheaper on English technical documentation may be more expensive on Chinese customer support transcripts.

The fix: Before committing to a provider, benchmark your actual production text through each candidate's tokenizer. The difference is measurable in hours. If you're doing 1M+ requests/day, a 10% tokenizer efficiency gap is thousands of dollars per month — more than enough to justify the benchmarking effort.

Leak #3: Prompt Caching — The 90% Discount Nobody Configures

Anthropic introduced prompt caching in August 2024. OpenAI followed with automatic caching in late 2024. Google Gemini launched context caching in early 2025. The discounts are substantial:

Now consider what's actually cacheable in a typical LLM application:

• System prompt: 500–2,000 tokens. Identical for every request. 100% cacheable.
• Few-shot examples: 500–3,000 tokens. Change rarely (when you update the prompt template). 100% cacheable between updates.
• RAG context: 2,000–8,000 tokens. Varies per query but often shares substantial overlap (same document chunks served to multiple users asking similar questions). Partially cacheable — 20–40% typical.
• Conversation history: 1,000–10,000 tokens. Unique per conversation. Not cacheable.

Real example: A customer support chatbot with a 1,500-token system prompt, 1,000 tokens of few-shot examples, and 3,000 tokens of retrieved knowledge base articles per query. Total input: 5,500 tokens per request. Cacheable: 2,500 tokens (45%). At 10,000 requests/day on Claude Sonnet 4, with caching configured:

• Naive annual cost: (5,500 × 10,000 × 365 / 1e6 × $3.00) + (800 × 10,000 × 365 / 1e6 × $15.00) = $60,225 + $43,800 = $104,025/year
• With caching: (3,000 × 10,000 × 365 / 1e6 × $3.00) + (2,500 × 10,000 × 365 / 1e6 × $0.30) + output = $32,850 + $2,738 + $43,800 = $79,388/year
• Savings: $24,638/year (24%) — from a single configuration change.

The fix: Identify your cacheable prefix tokens (system prompt, few-shot examples, static context). Structure your API calls so they appear at the beginning of every prompt — most caching implementations require the cacheable portion to be a prefix. Anthropic requires explicit cache point marking; OpenAI and Google handle it automatically for qualifying prefixes. Measure your cache hit rate in production and feed it into the LLM API Pricing Calculator to see the savings.

Leak #4: Batch Processing — Half Price If You Can Wait 24 Hours

OpenAI and Anthropic both offer batch (asynchronous) API endpoints with 50% off standard pricing. The tradeoff: up to 24-hour completion SLA instead of real-time response. For offline workloads — evaluation runs, dataset labeling, embedding generation, nightly summarization jobs, synthetic data generation — there is literally no downside. The 50% discount is free money.

Combined with prompt caching, the stackable discounts are dramatic. On Anthropic, a batch workload with 45% cacheable input tokens achieves an effective rate of:

• Cached input: 50% batch discount × 90% caching discount = 5% of sticker price
• Uncached input: 50% batch discount = 50% of sticker price
• Output: 50% batch discount = 50% of sticker price

For the customer support chatbot example above, adding batch processing where applicable (say 60% of requests are non-realtime — ticket summarization, weekly analytics, knowledge base updates):

• Optimized annual: $79,388 × (0.4 + 0.6 × 0.5) = $55,572/year
• Total savings vs naive: $48,453/year (47%)

The fix: Segment your LLM traffic into realtime and async. Realtime: user-facing chat, code completion, live translation. Async: summarization, evaluation, labeling, data processing, report generation. Route async traffic to batch endpoints. The infrastructure change is an API endpoint swap — no model changes, no prompt changes, no application logic changes.

Leak #5: Rate Limits and Retry Overhead — Paying Twice for the Same Token

Every LLM API has rate limits: requests per minute (RPM) and tokens per minute (TPM). When your application hits the limit, the API returns a 429 error. Most client libraries automatically retry — but the retried request consumes tokens again. A request that succeeds on the third retry consumed tokens three times for one successful output.

The cost of retries is not theoretical. For GPT-4o at $2.50/M input + $10/M output, each failed-and-retried 2,000-token prompt wastes $0.005 in input tokens. At 10,000 requests/day with a 2% retry rate, that's $1/day in wasted input tokens — $365/year. Small, but it compounds with scale.

The bigger cost is architectural: teams over-provision by deploying to multiple providers simultaneously to avoid rate limits, which means paying for capacity they don't need. Or they upgrade to higher rate-limit tiers (requiring prepaid commitments) before their usage justifies it.

The fix: Implement exponential backoff with jitter, not naive retry. Monitor your retry rate — if it exceeds 1%, you need either higher rate limits or a queuing layer. Batch endpoints have separate (and typically much higher) rate limits — moving async traffic to batch kills two birds: the batch discount and the rate limit headroom.

Provider-by-Provider Pricing: The 2026 Landscape

As of July 2026, the LLM API market has stratified into four tiers. The pricing below is public list pricing for on-demand API access. All prices are per 1 million tokens.

Tier 1: Premium Flagship ($10–$75/M output)

Anthropic Claude Opus 4 ($15/$75): The most expensive API by a wide margin. The 90% prompt caching discount makes it competitive for cacheable workloads — effective cached input price of $1.50/M is below GPT-4o's standard input price. For workloads with high cache hit rates and batch processing, Opus can be cheaper than GPT-4o despite the 7.5× sticker price on output. Best for: tasks where model quality is non-negotiable and caching is configured.

Tier 2: Standard Flagship ($2.50–$15/M output)

OpenAI GPT-4o ($2.50/$10): The reference point. Good performance across all task types. Automatic prompt caching (50% off) without explicit configuration. Batch API available. Best for: general-purpose workloads where you want a single provider and moderate cost.

Anthropic Claude Sonnet 4 ($3/$15): 50% more expensive than GPT-4o on output, 20% more on input. The 90% caching discount narrows the gap substantially for cacheable workloads. At a 40% cache hit rate, Sonnet is cheaper than GPT-4o for input-heavy workloads. Best for: cacheable workloads, code generation, long-context reasoning.

Google Gemini 2.5 Pro ($1.25/$5): The cheapest "flagship" tier model. 75% context caching discount. Largest context window in the industry (1M+ tokens). Best for: long-document processing, cost-sensitive workloads needing high capability.

Mistral Large 2 ($3/$9): Competitive with GPT-4o on price. Strong multilingual performance. No caching or batch discounts as of mid-2026. Best for: European deployments, multilingual workloads, sovereignty requirements.

Tier 3: Budget / Efficiency ($0.15–$1.25/M output)

OpenAI GPT-4o-mini ($0.15/$0.60): The budget benchmark. 4% of GPT-4o's output price for ~80% of the capability on structured tasks. Automatic prompt caching. Best for: classification, extraction, simple Q&A, high-volume filtering.

Anthropic Claude Haiku 4 ($0.25/$1.25): 2× GPT-4o-mini on output, but 90% caching discount. On cacheable workloads, competitive with GPT-4o-mini. Best for: cacheable lightweight tasks, multi-turn conversations with static system context.

Google Gemini 2.5 Flash ($0.15/$0.60): Price-matched to GPT-4o-mini. 1M token context window. Best for: long-document lightweight processing, competitive with GPT-4o-mini.

Llama 4 Scout via Groq ($0.15/$0.50): Open-weight model, lowest output price in the market. Groq's LPU inference delivers the fastest tokens/second. Best for: maximum throughput, lowest cost, real-time streaming.

Tier 4: Disruptor Pricing ($0.27–$2.19/M output)

DeepSeek-V3 ($0.27/$1.10): Flagship-capability model at below-budget-tier pricing. The price disruption that forced multiple rounds of cuts from OpenAI and Google. Chinese+English optimized tokenizer means English-only workloads may see slightly higher token counts. Best for: cost-sensitive workloads needing flagship capability, Chinese-language applications.

DeepSeek-R1 ($0.55/$2.19): Reasoning model. Even at 2× V3's price, it's cheaper than GPT-4o for reasoning-heavy tasks where it matches or exceeds flagship quality. Best for: math, code reasoning, complex analysis — tasks where reasoning quality justifies the premium over V3.

The Self-Hosted vs API Breakeven: When Does Buying GPUs Beat Paying Per Token?

The question every LLM-heavy startup asks at Series A: should we rent GPUs and self-host, or keep paying the API per token? The answer depends on model size, utilization rate, and which API tier you're comparing against.

The GPU Cost Baseline

GPU rental pricing (mid-2026, reserved instances, per GPU/month):

• NVIDIA H100 (80 GB): ~$1,800/month reserved ($2.50/hr), ~$3,000/month on-demand
• NVIDIA A100 (80 GB): ~$1,200/month reserved, ~$2,000/month on-demand
• NVIDIA L40S (48 GB): ~$800/month reserved

Model Sizing: What Fits Where

• 7–8B parameter model (Llama 4 Scout, Mistral Small): 1× H100 or 1× L40S. VRAM: ~14 GB in FP16, fits comfortably.
• 70B parameter model (Llama 4 Maverick): 2–4× H100 with tensor parallelism. VRAM: ~140 GB in FP16, requires multi-GPU.
• 405B parameter model (Llama 4 Behemoth): 8–16× H100. Not practical for most teams to self-host.

Breakeven Math: 8B Model

Self-hosted cost: 1× H100 reserved = $1,800/month. At 70% utilization (realistic for a well-tuned inference server with continuous traffic), this delivers approximately 1,500 tokens/second output. At 800 output tokens per request: ~6,750 requests/hour, ~4.86M requests/month.

API equivalent (DeepSeek-V3 at $0.27/$1.10, same token volumes): (4.86M × 2,000 input tokens / 1e6 × $0.27) + (4.86M × 800 output tokens / 1e6 × $1.10) = $2,624 + $4,277 = $6,901/month.

API equivalent (GPT-4o-mini at $0.15/$0.60): (4.86M × 2,000 / 1e6 × $0.15) + (4.86M × 800 / 1e6 × $0.60) = $1,458 + $2,333 = $3,791/month.

Against DeepSeek and budget APIs, self-hosting an 8B model wins at ~35% utilization (~2,400 requests/hour). Against GPT-4o-mini, self-hosting breakeven is closer to 50% utilization. Below that, the API is cheaper — and you don't manage infrastructure.

Breakeven Math: 70B Model

Self-hosted cost: 3× H100 reserved = $5,400/month. At 70% utilization: ~3,000 tokens/second output. At 800 output tokens/request: ~13,500 requests/hour, ~9.72M requests/month.

API equivalent (DeepSeek-V3): $5,248 + $8,554 = $13,802/month. Self-hosting wins at any utilization above 40%.

API equivalent (Claude Sonnet 4 at $3/$15): $58,320 + $116,640 = $174,960/month. Self-hosting wins at any utilization above 3%. The breakeven is essentially immediate for premium APIs at this scale.

The Utilization Reality Check

Most teams overestimate their GPU utilization. The theoretical breakeven is not the real breakeven. Self-hosted inference servers sit idle during traffic valleys (nights, weekends, holidays). The API charges zero for idle time. The three questions that determine which side of the breakeven you're on:

1. Is your traffic steady or bursty? Steady (continuous high throughput): self-hosted wins. Bursty (business hours only, spike-driven): API wins.
2. Can you batch requests? If yes, self-hosted utilization goes up and API costs go down (batch discount). The advantage shifts to API for most scales.
3. Do you have GPU infra expertise on staff? Self-hosting a 70B model across 3 GPUs requires understanding of tensor parallelism, quantization (AWQ/GPTQ/FP8), continuous batching (vLLM/TGI), and GPU node management. The engineering cost is real — allocate 0.25–0.5 FTE for a production self-hosted deployment.

Multi-Provider Architecture: Hedging Your Token Bets

Production LLM applications increasingly run across multiple providers — not for cost savings (though that's a side benefit), but for reliability. A single-provider architecture has a single point of failure: if the API is down, your application is down. Given that every major LLM provider has experienced multi-hour outages in 2025–2026, multi-provider is becoming table stakes for production deployments.

The architectural pattern is straightforward: a thin routing layer (LiteLLM, OpenRouter, or a hand-rolled proxy) that accepts OpenAI-format API calls and routes them to the configured backend. Fallback logic: try primary provider, if 429 or 5xx, retry on secondary, then tertiary. The routing layer also handles cost tracking per provider — essential for measuring whether your primary choice is still the right one.

Cost implications: Multi-provider architecture adds ~5–15% to your API bill (failed requests that get retried on a fallback provider, plus the overhead of routing). But it eliminates the far more expensive outage cost. The math is simple: if a single-provider outage costs you $X in lost revenue per hour, and the outage probability is Y% per year, multi-provider is worth it when the expected loss (X × Y) exceeds the multi-provider overhead. For most revenue-generating applications, the breakeven is unambiguous.

Multi-provider also enables cost-aware routing: send realtime requests to the cheapest provider that meets your latency SLA, send async batch requests to the cheapest period, and reserve the expensive flagship model for the 5% of requests that actually need it. A three-tier routing strategy (budget model for 80% of traffic, standard model for 15%, flagship for 5%) can cut your blended per-token cost by 60–80% compared to routing everything to the flagship.

Practical Decision Framework: Five Questions to Ask Before You Pick a Provider

1. What's your actual input-to-output token ratio? Measure it in production. Don't guess. The ratio is the single largest cost multiplier — and it determines which provider's pricing structure is cheapest for your specific workload.
2. What percentage of your input tokens are cacheable? Count your system prompt, few-shot examples, and static context. If it's above 20%, Anthropic's 90% caching discount may make Claude cheaper than GPT-4o despite the higher sticker price. Run the numbers — don't compare sticker prices, compare effective prices with your actual cache hit rate.
3. What percentage of your traffic can tolerate 24-hour latency? Offline processing, evaluation, labeling, report generation — anything that doesn't need a realtime response qualifies for batch pricing at 50% off. Moving 30% of traffic to batch cuts your blended cost by 15% with zero model quality impact.
4. Is your traffic steady or bursty? Steady, high-throughput traffic favors self-hosting (breakeven at surprisingly low volumes). Bursty traffic favors API (you pay zero for idle GPU time). Be honest about your utilization — most teams overestimate it.
5. Do you need multi-provider for reliability, not just cost? If yes, the cost conversation shifts: you're optimizing for blended cost across 2–3 providers with fallback logic, not the cheapest single provider. A three-tier routing strategy (budget/standard/flagship) yields the best blended cost for most production workloads.

Use the LLM API Pricing Calculator to model your specific workload across all 12 models from 6 providers, with caching and batch optimization. All calculations are client-side — your token counts and request volumes never leave your browser.

📜 Copyright & Attribution