Agent-first software comes down to one question: when the agent is confidently wrong, who finds out and how fast? That is the argument Abhi Arya of Reducto made at the AI Engineer World's Fair in 2026, after his team shipped an MCP server that broke the day the whole company used it.
Who pays when the agent is wrong?
Agent-first software, in Abhi Arya's framing, is defined by one question: who pays for the agent being wrong? Arya, who works on product at Reducto, an agentic document platform that turns unstructured documents into structured data, presented this at the AI Engineer World's Fair 2026 in San Francisco. He sorts every agent architecture into three buckets, and argues only one of them survives contact with real users.
The buckets are:
Auto mode, user-first, and agent experience compared
The three approaches differ in where autonomy sits and who absorbs the cost of mistakes. Arya argues the third bucket wins because it keeps the human in a position to verify.
| Approach | Where capability lives | Failure mode | Who pays for errors |
|---|---|---|---|
| Auto mode | The agent, with full autonomy | Confidently wrong, invisible to everyone | The builder, in front of customers |
| User-first | The human interface | Capped user understanding, weak agent | The user, through manual work |
| Agent experience in service of the user | Scoped tools with human verification | Bounded, inspectable mistakes | Shared, with the human able to catch errors |
Auto mode, he says, is what happens when you write a prompt and let the agent do whatever it wants: it will be confident about its mistakes, and you are the one answerable to customers. User-first gives the user every knob but builds on an agent that cannot do much, which he calls a chatbot experience. The third bucket structures the agent's capabilities around what the user actually wants, while leaving room for the human to check the work. This reframes agent experience from a question about the model into a question about context, capability, and self-improvement.
Why one MCP tool per API endpoint failed
Reducto's first MCP server, the Model Context Protocol standard for connecting agents to tools, mapped one tool to every API endpoint that drove the company's app, all in a single file. Arya built the MVP with Claude Code, Anthropic agentic coding tool that runs in the terminal, and shared it in the company's Slack. The curated demo worked. The company-wide rollout failed the same day.
The failure had a consistent shape. Someone fed the agent a half-finished description from a customer call or a Notion page. The agent then built an entire document pipeline end to end, reported that everything was correct, and neither the user nor the agent could explain what the workflow actually did. Arya's diagnosis: the team had built something that could communicate with their system, not something that could actually work with it. The problem was architecture, not prompt wording, so he deleted the whole file in one commit.
Fewer tools that carry the structure of real work
The rebuild replaced many low-level tools with fewer tools that encode how the product is meant to be used. The fix, in Arya's words, was moving context out of the prompt and into structure, so the agent stopped guessing and started referencing.
The concrete changes were:
- Chained domain tools. Instead of letting the model assemble a classify step and an extract step on its own, he built a single classify-to-extract tool, since routing Reducto's classification output into extraction is a known pattern. Tools chain automatically and the agent gets contextual guidance on what comes next after each step.
- A snapshot tool for live state. Agents are stateless, so one call hands back the pipeline's live state: the full flow context, plus surfaced errors with specific error codes. The agent references what exists instead of hallucinating variable names.
- Bounded shapes. Pipelines can only take the shapes the product engineers intended, so a human can follow exactly what the agent did and explain it to downstream users.
His summary: to avoid auto mode, do not hand your agent an API. Hand it tools that carry the right patterns, the state, and the validators, so the correct path is also the simplest path.
Making uncertainty visible at runtime
Legible construction fixed the building problem, but runtime failures still arrived in silence. A pipeline can run across 100 documents and quietly miss a scan on page 30, or a user can leave the document type unspecified. There is no segfault; the agent is simply confidently wrong.
Reducto's fix made uncertainty legible on two levels. First, outcome uncertainty: the company's Extract endpoint outputs confidence scores and bounding boxes, and tools feed that failure information back into the model to improve extraction schemas over time. This is a vendor-reported result from Reducto's own product, not an independent benchmark. Second, the agent's own uncertainty: in their evals, they began failing the agent for building on shaky assumptions, and added prompt guidance to use the ask-user-question tools that Claude Code and other frontier harnesses provide. The agent should say 'I can read this two ways, what did you mean?' rather than guess or stall.
The payoff compounds: one clarifying question delivers context, capability, and a human in the loop at the same time, and every pipeline the agent creates explains what it did and why so the user can review it.
Instrumentation that turned usage into a self-improving loop
The last layer was measurement. Every MCP session logged the prompts the model sent, and Arya asked the sales team to export their Claude Code sessions so the team could see where humans overrode the agent, where model confidence ran low, and where it asked questions.
That instrumentation flowed back into the MCP's long-term guidance, so the system got better at building pipelines by building more pipelines, without hand rewriting. It became a self-improving loop per organization and per user. Arya never told the sales team to use the agent. He exposed it as a product tool in Reducto's Cloud Enterprise offering, and people started wiring it into their own workflows: before a customer call, they would review notes from tools like CircleBack or Slack, then use the MCP to build a pipeline for that customer. The adoption signal he cites is cultural, not measured: people trusted the output enough to put it in front of high-value customers and to say they built it themselves.
FAQ
- What is agent-first software? In Abhi Arya's framing, it is an architecture question rather than a model question: design the agent's tools and interfaces around who pays when the agent is wrong. The winning pattern is agent experience in service of the user, where capabilities are scoped and a human can verify the work.
- Why does one MCP tool per API endpoint fail? Because the agent gets full autonomy over raw endpoints and assembles workflows nobody can explain. When it builds something wrong, it reports success confidently, and there is no structure for a human to inspect. The fix is fewer tools that encode the correct workflow patterns.
- What is a snapshot tool in an MCP server? It is a single tool call that returns the live state of the system, including flow context and surfaced error codes. Because agents are stateless, this lets the model reference what actually exists instead of guessing names or hallucinating.
- How do you stop an AI agent from guessing? Reducto made uncertainty legible: confidence scores and bounding boxes fed back from extraction results, eval failures for building on shaky assumptions, and prompt guidance to use ask-the-user tools when input is ambiguous. One clarifying question adds context, capability, and human oversight at once.
- Who is Abhi Arya? He works on product at Reducto, a company that describes itself as an agentic document platform and reports processing over 3 billion documents in three years for customers including Harvey and Scale AI. He spoke at the AI Engineer World's Fair 2026 in San Francisco.
From conference talk to readable article
The core lesson of this talk applies to content as much as to architecture: knowledge that stays locked in one format, a video here, a live demo there, rarely reaches the people who need it. If you have talks, interviews, or explanations sitting in YouTube videos, Skala Blog turns them into written articles: paste the YouTube URL, the video gets transcribed, and you get an editable draft built for readers and search. The recording keeps the insight; the article makes it findable.
Fork this article
Start a new branch from the same video, shaped your way. You keep the credit; the original keeps the attribution.
A fork in another language is filed as a translation of this article, so the two pages point at each other. You can unlink it later from the editor.
0/240
You are creating
- Format
- For
- Language
- Source
- Your angle
You will be asked to sign in before it is generated.
Buy credits