Cheap Inference Just Grew a Clock
Cheap Inference Just Grew a Clock
Thesis: The lab that trained operators to treat frontier tokens as a flat discount just switched on peak and off-peak. Cache-hit input on its production model jumps as much as 12x. Cheap inference is no longer a unit price. It is time-of-day inventory. An AI-native commerce company that cannot schedule agent work, discipline cache, and refuse peak for batch jobs will pay rush-hour freight for overnight merchandising, support summarization, and shopping-agent compare loops.
I am building an AI-native commerce company. I want agents in catalog, support, and the buyer’s cart. I refuse to run them on yesterday’s flat card and call the overrun “AI strategy.”
The Signal: The Clock Became a Price
Overnight heat was not another leaderboard screenshot. It was a tariff.
DeepSeek’s own API docs now say the quiet part in a footnote: from 16:00 UTC on August 16, 2026, V4-Flash and V4-Pro leave the single list price and enter peak / off-peak billing. Peak windows are 01:00-04:00 and 06:00-10:00 UTC. Off-peak is half of peak — which is not a gift. It is half of a much higher card.
The numbers that should sit on a merchant P&L, not a model-fan thread:
| Line | Yesterday (flat) | Off-peak | Peak |
|---|---|---|---|
| V4-Pro output / 1M | $0.87 | $1.98 | $3.96 |
| V4-Flash output / 1M | $0.28 | $0.66 | $1.32 |
| V4-Pro cache-hit input / 1M | $0.003625 | $0.022 | $0.044 |
Pro output is about 2.3x yesterday off-peak and about 4.6x at peak. Cache-hit input is the shock: roughly 6x off-peak and 12x at peak. Builder X compressed it cleanly: if your loop assumed yesterday’s flat card, you do not have a pricing opinion. You have a timezone problem.
Peak is China workday. US Eastern daytime is mostly off-peak. US overnight — the window operators love for catalog rebuilds, feed pushes, and “let the agent cook” — sits inside peak. Do not park a night batch there and call it cheap.
I already argued that you should buy inference like inventory and that one frontier model for every step is a margin tax. Stack the next layer. Budget and routing were about how much and which model. The new scarce control is when. A token that used to be a constant is now a lane: congested, priced, and hostile to the jobs you scheduled for “when nobody is shopping.”
| What the market still says | What a merchant P&L should hear |
|---|---|
| ”DeepSeek is still cheaper than Opus” | Peak Pro output is a different SKU than yesterday’s flat |
| ”Cache hits make long loops free” | Cache-hit input just became the tax on chatty agents |
| ”Run the batch at 2am, save money” | 2am US is often peak UTC. You bought rush hour |
| ”We’ll pick a cheaper model later” | The clock applies before the model swap |
Cache-Heavy Loops Are the Tax, Not the Save
Commerce agents are cache machines wearing a brand. A support summarizer re-sends the same policy block every ticket. A merchandising agent re-attaches the same attribute schema every SKU. A buyer-side compare loop carries the same “navy polo, ships Friday, under $80” prompt across twenty tool results. Yesterday that repetition was the discount: cache-hit input on Pro sat near a third of a cent per million tokens. Today the same habit is a meter. The more your loop looks like a real store — long context, repeated tools, a constitution that must travel with every turn — the more the new card punishes you.
We treated caching as the adult move: stuff the catalog contract into the prefix, hit cache, feel sophisticated. That was correct when cache was nearly free. It is malpractice when cache-hit input on the production model rises more than a thousand percent at peak. A “well-cached” overnight enrich job is now the job most exposed to the hike.
Do not answer the hike by dumping work onto a “free” router. Overnight chatter is celebrating billion-token giveaways. The honest posts say the quiet part: a shared free pool is a routing-data harvest. Every prompt teaches someone else’s layer which model wins. If those prompts contain live price, MAP, return windows, or customer context, you did not save COGS. You leaked the catalog to pay for someone else’s preference engine.
You cannot move the buyer’s clock. You can move the work that is not the buyer: feed rebuilds, alt-text, attribute cleanup, nightly reprice proposals, ticket clustering. Live checkout and live support may pay peak. Batch that masquerades as live should not.
Think of it as freight, not software. A 3PL does not quote one rate for “a box.” It quotes a lane, a cutoff, and a peak surcharge. Inference just grew the same UI. Founders who still model agent COGS as tokens x brochure price are quoting LTL as if it were a stamp.
Operator Playbook: Think Big, Step Small, Do Smart
Think big. Inference is becoming a time-of-day market, the way ads, energy, and outbound freight already are. The company that wins will not pick a model and pray the price war returns. It will own a schedule + cache policy + kill criteria the way a merchant owns safety stock. Preference still goes to the executable catalog. Margin goes to whoever refuses to enrich that catalog at the congested hour.
Step small (Monday morning). Do not rebuild the stack. Put a clock on one surface.
- Inventory the loop against UTC, not “overnight.” List live support, live quote, catalog enrich, feed rebuild, reprice, fraud pre-screen. Tag each live (someone is waiting) or batch (the world can wait six hours). Write the UTC window. If you cannot name it, you do not have a cost model.
- Move batch off the China workday peaks. Peak is 01:00-04:00 and 06:00-10:00 UTC. That eats a slice of US night. Live checkout stays live. Nightly merchandising is not live. If a job can wait, it must wait.
- Measure cache-hit share of spend, not hit rate as vanity. A 90% hit rate can now mean you concentrated the hike on the prefix you re-send every turn. Put stable policy behind a quote function, not a 40k-token system prompt.
- Give live and batch different kill criteria. Live support may spend peak Pro if first-contact resolution holds. A batch enrich that burns peak Flash on SKUs nobody will sell this week is spoilage. Cap dollars per peak hour and tokens per accepted SKU.
- Do not flee into a harvest. If a router is free, assume your prompts train their ranker. Strip PII, MAP, and unpublished price from anything that leaves your VPC.
Do smart. Route by hour the same way you already should route by task. Cheap specialists for high-volume enrich. Frontier only for hard judgments. Off-peak for anything that is not a waiting human. Verification still sits in front of money: a cheaper, better-timed loop that writes a refund without a receipt is still a leak.
This week, take one nightly job — top 200 SKU cleanup, or yesterday’s ticket summaries — and answer four questions before you buy another model: What UTC window does it run? Live or batch? What share of the bill is cache-hit input? What happens if we delay it four hours? If you cannot answer, you are renting congestion.
The Claim Worth Arguing
A flat token price is not a cost model. Cheap inference just grew a clock. Merchants who cannot schedule the loop will pay peak freight for batch work.
The counterexample I want: a team that kept every agent job on one always-on frontier card, ignored peak windows, let cache-hit input dominate the bill, and still beat a competitor who scheduled batch off-peak. If that exists at scale, I want the hourly invoice.
Until then, I will build as if labs keep growing clocks, cache stops being a stealth discount, and the operator who owns the schedule owns the margin.
If you disagree, bring the counterexample on X. Best failure mode wins — especially if your cheapest job ran at peak and you called it overnight savings.
Sources
- DeepSeek API docs: Models & Pricing (peak / off-peak effective 16:00 UTC, 16 Aug 2026)
- Secondary (X, 15-16 Aug): builder breakdowns of the V4 cache-hit hike, China-workday peak windows, and free-token routers as routing-data harvests