Long-Running Is Not Long-Horizon

Thesis: A full-year e-commerce simulation just scored the best frontier agent setup at 27.3% of the mean final net assets that humans achieved — and those humans were not lifelong merchants. That is not a leaderboard insult. It is a product map. Long-running is not long-horizon. Agents can stay online for 365 simulated days and still stop acting, shrink their control loops, ignore delayed refunds and penalties, and reinforce bad memory. The AI-native commerce company that wins will not sell “fully autonomous store managers.” It will own delayed-feedback architecture with humans still holding capital allocation.

I am building an AI-native commerce company. I will take a human who owns margin, assortment, and kill criteria over an agent that can run forever while the P&L quietly dies.

The Signal: A Year of Store Ops, Not a Demo

The seed is concrete. Alibaba’s MerchantBench (arXiv 2607.28956) is a 365-day, order-level simulation grounded in 98,843 real product records and 26 merchant tools — sourcing, pricing, order tracking, cash-flow, supplier shocks, refunds, negative reviews, penalties. Eight LLMs, two agent frameworks, 48 full-year runs. Best configuration: 27.3% of the mean final net assets of the human baseline. The humans, notably, were three participants without prior e-commerce operating experience.

That last detail should make every “AI store manager” pitch slide uncomfortable.

Hugging Face paper chatter and builder X amplified the number because it is rare to see commerce evaluated on end-state capital, not chat quality. Parallel discourse this week is still selling autonomous shopping assistants (retail brands shipping Google Cloud agents) and “product feed as storefront” stacks. Both matter. Neither answers MerchantBench’s question: after three hundred sixty-five days of compounding decisions, who still has the cash?

I already wrote that one-shot IQ does not run a store and that verification is the moat. Stack the next layer. Continuity across a few days is necessary. Continuity that still compounds capital under delayed outcomes is the actual merchant job. MerchantBench’s authors put it cleanly: long-running does not necessarily mean long-horizon. Agents may gradually stop acting, narrow control loops, fail to respond to delayed feedback, or reinforce incorrect assumptions through memory.

Founder translation: if your autonomy demo ends when the video ends, you measured endurance of process, not ownership of P&L.

Why Agents Lose the Year

Commerce is a control problem with lag.

Failure mode (from the sim pattern)	What it looks like in a real store	Monday morning risk
Stop acting	Agent “monitors” and never re-prices or reorders	Stockouts and silent margin death
Narrow control loop	Only fixes today’s tickets; ignores cash and supplier	Local wins, global loss
Delayed feedback blind spot	Refunds, chargebacks, reviews, penalties land weeks later	Happy-path week, wrecked quarter
Memory that lies	Wrong MAP rule or supplier lead time gets “learned” hard	Confident error at machine speed

A model can draft a beautiful promo on day 12 and still be a terrible merchant by day 212 if it never closes the loop on what that promo did to returns, review score, and contribution margin. Humans who have never run a store still beat frontier agents on year-end net assets because they keep a global objective in their head: do not go broke; do not freeze; update beliefs when delayed pain arrives.

That is not mystical intuition. It is horizon discipline.

Think about the unit economics. An agent that takes 40 tools calls a day for a year is not free just because tokens got cheaper. If half those calls re-explain the catalog, half ignore last month’s penalty, and the write path still has no kill criteria, you are buying expensive amnesia with occasional brilliance. I argued you should buy inference like inventory. MerchantBench is why: the scarce resource is not another tool in the agent belt. It is a control system that still cares about cash after the novelty wears off.

Also notice what the benchmark does not reward: brand theater, homepage polish, or in-chat checkout demos. It rewards operating under incomplete information with irreversible inventory and money. That is the same world where credentialed shopping agents show up as buyers, where machine-readable feeds win selection, and where an agent with a wallet and no receipt is fraud surface. Different surfaces. Same law: actions that move capital need delayed accountability.

Product Map: What 27% Is Asking You to Build

Do not read MerchantBench as “wait for smarter models.” Read it as a spec for the product layer models will still need.

1. Global objective that survives session boundaries.
Net assets, contribution margin, cash runway, SLA breach rate — pick numbers an agent cannot “locally optimize” into a dead store. Put them in the system of record, not in the system prompt.

2. Delayed-outcome queues as first-class objects.
Refunds, chargebacks, negative reviews, supplier late fees, MAP violations. These arrive after the happy path. If your agent architecture only scores the day of the action, you are training it to ignore the bill.

3. Actuation health, not chat health.
Instrument: days with zero pricing updates, days with zero reorder decisions, days with tool-call volume but no capital-relevant write. “Still talking” is not “still merchandising.” Stopping action is a failure mode, not a quiet success.

4. Memory with audit and expiry.
Wrong beliefs that compound are worse than no memory. Every durable merchant fact needs owner, source, timestamp, and a way to kill it. Memory that cannot be corrected is how agents reinforce incorrect assumptions for 300 days.

5. Human capital allocation with agent proposal rights.
Let agents propose assortment shifts, price bands, and supplier switches. Keep a human (or a tightly scoped policy engine with a named owner) on capital commitments above a threshold. Autonomy below the line. Authority above it.

Think big: in five years, serious merchant platforms will ship a horizon controller the way they ship OMS today — a layer that forces long-horizon objectives into short-horizon agent steps.

Step small: you do not need a 365-day sandbox on Monday. You need one delayed feedback loop wired end to end.

Do smart: pick the loop where money or inventory already hurts when you ignore lag.

Monday Morning Playbook

If you run a store or an AI-commerce product, do this week — not next quarter:

Name three delayed outcomes that currently have no agent owner (e.g. refund rate by promo cohort, supplier penalty by PO, review score by SKU family). Put each on a weekly review with a human merge owner.
Define one capital threshold. Above it, agents may only propose. Below it, agents may execute with receipts. Write the number down. Share it with eng and finance.
Add an actuation dashboard. Count capital-relevant writes per day (price change, reorder, cancel, refund decision). Alert when the agent is “busy” in tokens but flat in writes for 48 hours.
Kill one false memory path. If policy lives in Slack or a chat summary, move one MAP or return rule into structured storage with version and owner. That is the smallest continuity win that compounds.
Refuse the “autonomous GM” demo until you can show a 30-day paper trade against a human baseline on the same assortment — net assets, not vibes. MerchantBench already told you the default outcome of unscoped year-long autonomy.

Operators who do this will still use frontier models. They will just stop confusing runtime with horizon.

The Claim Worth Arguing

A year of store ops is a capital control problem. Agents that only maximize today’s tool success will lose to humans who maximize next quarter’s net assets — until product architecture forces delayed feedback into every write path.

Counterexample I want: a merchant who ran agent-led assortment and pricing for 90+ days, beat a human baseline on contribution margin, and can show the delayed metrics (refunds, penalties, review) did not secretly reverse the win. Bring the trajectory, not the launch tweet.

If you are shipping AI into commerce and you think 27% is “just a benchmark,” tell me why your production agent will keep acting — and keep owning cash — when the refund lands in week six.

Disagree on X. Best counterexample wins.

Sources

MerchantBench paper (arXiv): https://arxiv.org/abs/2607.28956
Hugging Face papers index / discussion of MerchantBench results (public X + community summary of the 27.3% human net-assets gap)
Retail AI shopping assistant launches (e.g. Dunelm + Google Cloud) as parallel “autonomy” product heat — useful contrast, not the thesis seed