Reliable AI Agents Need Two Modes: Workflows and Verification Loops

Most AI agent demos fail at the exact moment real work begins: step seven.

The first six steps look magical. The model understands the request, calls a tool, finds a document, updates a record and writes something clever. Then one small assumption is wrong, the next action compounds it, and the agent confidently announces that the job is done.

That is not an intelligence problem. It is an architecture problem. Reliable AI agents need two different operating modes: deterministic workflows for the path you already understand, and verification loops for the parts where the model must explore, judge and prove completion.

The autonomy trap

We have made “autonomous” sound like the highest possible compliment for software. It is not. Autonomy is useful when the route cannot be known in advance. Everywhere else, it is often a very expensive way to replace a checklist.

Anthropic draws the useful distinction: workflows move models and tools through predefined code paths, while agents dynamically decide their own process and tool use. Their practical advice is refreshingly unsexy: start with the simplest solution, then add complexity only when it earns its latency and cost.

I agree. If a process always means “fetch these companies, clean these pages, extract these fields, validate the output and write the report,” letting an agent reinvent that route on every run is not flexibility. It is variance wearing a clever hat.

Use a deterministic workflow when the path is known

A deterministic workflow does not mean every step is dumb code. Models can still classify, extract, summarize and decide inside a bounded stage. The difference is that the surrounding system owns the sequence, the allowed tools, the data contract and the failure handling.

This is the right default for long, repeatable processes with clear inputs and outputs. Company enrichment is a good example. You may need model judgment to select relevant text or extract people from messy pages, but the pipeline itself should not wake up each morning and wonder what enrichment means.

Fixed stages also make evaluation possible. You can compare cleaners, chunk sizes, model tiers and fallback providers without changing five variables at once. When quality drops, you know where to look. When costs rise, you can attribute them to a stage instead of shrugging at one giant agent trace.

This matters because reliability is not one score. It is a chain. Selection recall, extraction precision, tool failures, malformed outputs and duplicate records can each look acceptable alone while producing a bad final result together.

Use an agent loop when the definition of done needs judgment

Some jobs do not have a fixed route. “Find why this deployment is failing and fix it” may require reading logs, inspecting code, running tests, changing a hypothesis and trying again. A rigid workflow can become absurdly complex because it has to encode every possible branch.

That is where an agent earns its autonomy. But the loop needs a goal, constraints and an external definition of done. The agent should plan, act, observe the result, evaluate the evidence and continue until the acceptance criteria pass or a human decision is required.

This is close to the self-directed loop Anthropic describes in its trustworthy-agents work: plan, act, observe, adjust and repeat. Crucially, the same research stresses that useful agents still need human control, calibrated check-ins and safeguards across the model, harness, tools and environment.

“Done” is a claim. Evidence is the product

The most dangerous sentence in agent software is “Task completed successfully.” It sounds like proof, but it is just another model output.

A reliable system asks for evidence the agent cannot fake with confidence. For code, that may be passing tests, a clean type check and a diff constrained to the requested files. For research, it may be source coverage, dates and contradiction checks. For data enrichment, it may be schema validation, deduplication, reachable-contact rates and a review of the costliest outliers.

OpenAI’s guidance now treats this as an engineering discipline rather than prompt decoration. Its agent-evaluation guide recommends reproducible evaluations, while trace grading scores the decisions, tool calls and reasoning path inside a run. That distinction is important: a lucky final answer can hide a terrible process.

The evaluator also needs calibration. If it is too lenient, the loop exits early. If it is too strict, the agent burns money polishing work nobody asked for. Start with explicit acceptance criteria, test them against human-reviewed examples, and track false passes as seriously as task failures.

Route intelligence by stage, not by ego

Not every step deserves your most expensive model. Planning under uncertainty may need high intelligence. Cleaning text, applying a known transformation or checking a simple schema usually does not.

This became even more relevant on July 30, 2026, when OpenAI cut GPT-5.6 Luna pricing by 80%. The useful lesson is not “use the cheap model everywhere.” It is to rerun your evaluations. Old cost assumptions expire the moment model pricing or performance changes.

The strongest architecture often uses a more capable model to resolve ambiguity and define the plan, then a faster model to execute well-specified stages, run tests and evaluate routine outputs. OpenAI itself gives a similar coding example in that pricing announcement. Model routing should be measured against the outcome, not chosen from a leaderboard screenshot.

The architecture I would ship

For a serious multi-step system, I would put a deterministic workflow around the stable business process and insert agent loops only where the route genuinely requires discovery. Every loop would have a budget, a stop condition, approved tools and evidence-based acceptance criteria. High-impact actions would require human approval. Every run would leave a trace that can be evaluated later.

That is also the product direction I find most useful for Notis: Workflows for repeatable sequences, and Goals for jobs where the AI intern must keep working until it can prove the result meets the brief. The user should not have to watch every click, but the system should never confuse silence with success.

The next time someone proposes “one autonomous agent” for a 20-step operation, ask a less glamorous question: which steps actually need autonomy?

That question will save more money, debugging and awkward demos than another clever prompt ever will.