One Frontier Model for Every Step Is a Margin Tax
One Frontier Model for Every Step Is a Margin Tax
Thesis: Agentic commerce is not won by wiring one frontier model into every workflow step. It is won by treating the agent as a system of models: route high-volume, low-risk work to cheap specialized models, reserve frontier IQ for planning and hard judgments, and post-train where your order data makes a specialist beat a generalist. One-model architecture is a margin tax wearing a demo costume.
I am building an AI-native commerce company. I want agents in catalog, support, fraud pre-screen, and merchandising. I refuse to pay “best lab of the week” prices for steps that do not need a philosopher.
The Signal: Routing Became the Product Story
Overnight heat on X and in the open-source stack was not another leaderboard. It was orchestration.
NVIDIA’s NeMo Switchyard is being framed as what production agents actually need: an open model router that reshuffles which model handles each step mid-task. The pitch is blunt — match the right model to the right step of a workflow, and task cost can fall hard (their own tests talked about cutting cost to roughly a third) while quality holds or improves. Beside it sits Nemotron Lightning-class open models aimed at always-on specialized agent tasks, with explicit guidance to send high-volume execution to the small efficient model and keep frontier capacity for complex planning.
The same discourse includes post-training stories that matter more for operators than another parameter count. Open weights post-trained for a few hours on a real customer-support workload can outperform a general frontier model on that job — because the job is narrow, the data is yours, and the evaluation is acceptance rate, not a public chat bench.
Founder translation:
| What the market celebrates | What a commerce P&L should hear |
|---|---|
| ”System of models” routers | Default every-step frontier API is optional |
| Open MoE models for agents | High-volume steps can leave the blank-check meter |
| Six-hour post-train on support data | Specialist beats generalist when the task is stable |
| One smart agent demo | One model for enrich, triage, dispute, and plan is COGS malpractice |
I already argued that you should buy inference like inventory — caps, turns, spoilage, kill criteria. This essay is the architecture sibling. Budgeting without routing is still writing one blank check and hoping the agent is polite about it.
Commerce Steps Are Not Equal Intelligence Problems
A storefront agent trajectory is not “one question.” It is a pipeline of different risk and IQ requirements:
- Parse and retrieve — title normalize, attribute extract, policy lookup, ticket classify
- Propose — rewrite product copy, suggest a substitute SKU, draft a support reply
- Plan — multi-day promo structure, inventory rebalance hypothesis, multi-party dispute narrative
- Decide / write money — issue refund, capture payment, cancel hold, change MAP price
Steps 1 and most of 2 are high volume and low irreversibility. They should run on the cheapest model that hits a measured acceptance bar. Step 3 sometimes needs frontier reasoning. Step 4 should barely care which model drafted the proposal — it should care about authority, verification, and a human or hard gate, which is the stack I already argued in Verification Is the Moat and Never Give a Shopping Agent a Wallet Without a Receipt.
When teams ship “our agent uses Model X,” they collapse those layers. Catalog enrichment burns the same API key as refund judgment. L1 triage burns the same as multi-order fraud narrative. The model gets smarter; the unit cost per accepted task does not, because you never separated the steps that create volume from the steps that create liability.
That is why router heat is a commerce story, not only an infra story. Routing is how you encode:
- Cost class per step (cheap / mid / frontier)
- Latency class (always-on enrichment vs. async plan)
- Authority class (read-only propose vs. write with receipt)
- Eval class (auto-accept if schema validates vs. merge owner required)
Without those classes, “agentic” just means unstructured inference spend attached to every button.
Specialists Are the Moat Frontier Models Cannot Own
Open weights and post-training compress the gap between “we use the best closed API” and “we own a worker for this job.” That does not make distribution free. It does make one-size frontier product a worse default.
What you can specialize with merchant data:
| Surface | What to post-train / route for | What stays general |
|---|---|---|
| Catalog | Attribute extraction, policy-to-field mapping, title hygiene | Brand voice for hero campaigns |
| Support L1 | Your return windows, shipping exceptions, tone, tool schemas | Novel multi-order legal disputes |
| Merchandising | MAP-safe rewrite, channel-specific claims | Category strategy and capital allocation |
| Fraud pre-screen | Your historical chargeback patterns and device signals | Novel coordinated attacks (escalate) |
The moat is not the base Nemotron (or peer) weights. Those commoditize on purpose — that is the point of open families and routers. The moat is the loop: labeled accept/reject outcomes from your merge owners, order-event trajectories, policy changes, and the eval harness that says “this specialist is allowed to propose refunds only inside these SKUs and amounts.”
Do smart: if you cannot measure first-pass acceptance for a surface, you are not ready to post-train it. You will overfit a vibes dataset and ship confident wrong answers faster — which is how more support agents industrialize error.
Think big: in two years, competitive AI-native commerce stacks look less like “Claude everywhere” and more like a portfolio with a routing policy and receipts. Cheap models for machine-readable catalog hygiene. Specialists for L1. Frontier only when the trajectory is long-horizon or high-stakes. Verification always at the money line.
Monday Morning Playbook
Think big. Stop asking “which model should run the company?” Ask “which step deserves which model, budget, and write authority?” Architecture is the product; model names are inventory SKUs.
Step small — one experiment this week:
- Pick one agent surface already in production or pilot (catalog field fill, L1 draft replies, or fraud pre-tags).
- Split the trajectory into three buckets: retrieve/classify, propose, money-or-policy write.
- Re-route bucket 1 to a small/open/cheap model with a hard schema check (JSON fields, allowed enums, max length). Keep your current frontier model only for bucket 2 if needed. Bucket 3 never auto-executes without your existing authority gate.
- Measure for seven days: cost per accepted task, first-pass acceptance rate, human edit distance, and any increase in refunds/chargebacks or bad catalog publishes.
- Kill or promote: if cost drops and acceptance holds, expand routing. If acceptance falls, fix the schema and evals before you “add more intelligence.”
Do smart: write the routing policy as config, not as tribal knowledge in a Slack thread. Name the models by role (catalog-extract-v1, support-draft-v2, dispute-plan-frontier) so swapping a lab does not rewrite your product. Cap weekly spend per role the way you cap inventory buys for a test SKU.
Optional second experiment if you already have labeled support tickets: take 200 accept/reject pairs and run a short post-train or preference pass on an open small model. Compare against frontier drafts on the same eval set. If the specialist wins on acceptance and cost, promote it behind the same verification rails. If it loses, you still learned that your task is not as “specializable” as the pitch deck claimed — which is valuable before you rent a GPU cluster.
The Claim Worth Arguing
The scarce skill in AI-native commerce is no longer access to a smart model. Access is the default. The scarce skill is routing discipline: putting the right intelligence, cost, and authority on each step so machine demand does not eat contribution margin.
If your counterexample is “we only use one frontier model and margins are fine,” show cost per completed task with retries included, not the brochure $/M tokens. If your counterexample is “routers are premature optimization,” show a catalog or support agent where every step truly needs the same IQ and the same risk class — I have not found one in a real store.
I will keep shipping machine-readable catalogs, verification gates, and adversarial rule design. I will also ship model portfolios — because one frontier model for every step is not cautious. It is lazy COGS.
Disagree? Best counterexample wins — a merchant who ran every agent step on the same frontier API, scaled volume, and still improved contribution margin. Find me on X and argue it.
Sources
- NVIDIA NeMotron / NeMo Switchyard (open model routing for agent workflows; Lightning for specialized high-volume steps): https://developer.nvidia.com/topics/ai/nemotron
- NeMo Switchyard repository: https://github.com/NVIDIA-NeMo/Switchyard
- VentureBeat coverage of NVIDIA Switchyard mid-task model routing and cost claims (secondary report of vendor tests): search “NVIDIA Switchyard router VentureBeat”