Single Agent or Multi-Agent System? How to Decide (and When It's Overkill)

Somewhere in the last year, "multi-agent" stopped being an architecture choice and became a status symbol. Architecture diagrams grew boxes. Every workflow got a researcher, a planner, a critic, and a manager, each with a name and a personality.

Then the system went to production, and someone had to figure out why it had cost four times the estimate, taken forty seconds to answer, and produced a confident wrong answer that none of the five agents flagged.

We're not against multi-agent systems. We build them. But in our experience the more common mistake isn't using one when you should have used a single agent with good tools. It's the reverse of what the diagrams suggest: reaching for complexity before the simple version has been pushed to its limit.

Single Agent vs. Multi-Agent System: The Short Answer

A single agent is one LLM-driven loop that plans, calls tools, and produces the result. A multi-agent system splits the work across several agents, usually an orchestrator that delegates to specialists such as research, drafting, and compliance checking. Start with a single agent and move to multiple agents only when you hit a specific limit: context overload, genuinely parallel work, or a need for independent review.

That last clause is the whole article in one sentence. The rest is how to tell whether you've hit one of those limits.

Key Takeaways

Default to the simplest thing that works: a single agent, or even a fixed workflow with LLM steps. Add agents when a measured problem justifies it.
Multi-agent systems cost more. Anthropic reported its multi-agent research system used roughly 15 times the tokens of a chat interaction, versus about 4 times for a single agent.
Debugging gets harder with every agent boundary, because failures hide in the handoffs.
Reliability compounds against you in sequential chains. Small error rates multiply.
Multi-agent earns its keep in three cases: parallelizable breadth, tasks that overflow one context, and work that needs independent review.
A compliance check is often the best first candidate for a second agent, because independence is the point.

What We Mean by Each Pattern

Terms get slippery here, so let's pin them down.

Single agent. One model, one system prompt, a set of tools, and a loop: think, call a tool, read the result, repeat until done. It can still be sophisticated. It might have fifteen tools, long-term memory, and retrieval behind it. "Single" refers to the number of decision-making loops, not the capability.

Orchestrator plus specialists. A coordinating agent receives the task, breaks it down, and hands pieces to specialist agents, each with its own instructions, tools, and often its own context. Results flow back to the orchestrator, which assembles the final output.

Workflow (the middle option nobody draws). A fixed sequence you define in code: step one calls the model to extract facts, step two calls it to draft, step three runs a check. The model fills in steps, but your code decides the order. This isn't an agent at all, and for many business processes it's the right answer.

Here's a running example. A team produces customer-facing proposals. The job involves three kinds of work: research (pull account history, pricing, and relevant case material), drafting (write the proposal in the company's voice), and compliance checking (verify claims, required disclosures, and approved terms).

We'll use it to compare the options.

SINGLE AGENT                       ORCHESTRATOR + SPECIALISTS

   User request                       User request
        |                                  |
        v                                  v
  +-------------+                   +--------------+
  |   Agent     |                   | Orchestrator |
  |  (all tools)|                   +--+-----+---+-+
  +-------------+                      |     |   |
   research tools                      v     v   v
   drafting                        Research Draft Compliance
   compliance rules                 agent   agent   agent

Head-to-Head Comparison

	Single agent	Orchestrator + specialists
Build effort	Low	Medium to high
Token cost per task	Lower	Typically much higher
Latency	One loop (can still be slow)	Faster if parallel, slower if sequential
Debugging	One trace to read	Multiple traces plus handoffs
Reliability on long chains	Fewer handoff failures	More points where errors compound
Context handling	Can overflow on big tasks	Each specialist gets a focused context
Independent review	Weak (it checks its own work)	Strong (a separate agent can audit)
Parallel work	Limited	Natural fit
Best for	Focused, sequential tasks	Broad, parallelizable, or audited tasks

Read this table as trade-offs, not a scoreboard. Every row where multi-agent wins has a cost somewhere else.

The Cost Trade-Off

Let's start with the one that surprises people most.

Every agent boundary multiplies tokens. The orchestrator reads the task and writes instructions. Each specialist receives context, often a duplicate of what others already saw, does its work, and returns results. The orchestrator then reads all of that and synthesizes it. You're paying for the same information to be read several times.

There's one well-documented data point. In June 2025, Anthropic's engineering team described its multi-agent research system and reported that agents typically use about 4 times more tokens than chat interactions, while multi-agent systems use about 15 times more. The same write-up reported that the multi-agent setup beat a single-agent baseline by 90.2% on their internal research evaluation, and that token usage alone explained most of the performance difference on one of the benchmarks they analyzed.

Read that carefully, because it cuts both ways:

It shows multi-agent can deliver meaningfully better results on the right task.
It also suggests a good part of the gain came from spending more compute, which a single agent given a bigger budget might partly match.
And it's one system on one kind of workload. Treat 15x as an illustration, not a constant. Measure your own.

Anthropic's own conclusion was that this only makes economic sense when the task is valuable enough to justify the spend.

What this means for budgeting: For the proposal example, a single agent might run through research, drafting, and checking in one pass. Splitting into three specialists means three separate context loads plus orchestration overhead. If the proposal is a high-value enterprise deal, the extra cost is trivial. If you're generating a thousand routine quotes a day, it isn't.

A practical habit: before choosing, estimate cost per task for both designs, multiply by your volume, and compare against what the task is worth. It's a short spreadsheet, and it ends a lot of arguments.

The Debugging Trade-Off

With one agent, when something goes wrong you read one trace. Prompt in, tool calls, outputs, final answer. It's tedious, but it's linear.

With multiple agents, the failure could be in any of the following places:

The orchestrator misunderstood the task
It gave a specialist unclear instructions
The specialist did the job correctly but returned something the orchestrator misread
Two specialists produced conflicting results and nothing resolved it
Context got summarized between agents and a crucial detail dropped out

That last one is the sneaky one. Information passing between agents is usually compressed. The compliance agent may never see the detail in the research notes that made a claim risky, because the orchestrator summarized it away. Nobody made an error you could point to. The system just lost information at a boundary.

Some teams, including the group behind the Devin coding agent, have publicly argued that this context-sharing problem is a main reason to be wary of multi-agent designs, especially where agents make dependent decisions that need the same full picture.

What to do about it, if you go multi-agent:

Trace everything with a shared ID. One request, one trace, every agent's calls nested inside it.
Log the handoffs, not just the outputs. What exactly did the orchestrator send? What exactly came back?
Keep handoffs structured. Pass defined fields (client name, claims to verify, sources) rather than free-form summaries.
Make agents individually testable. If you can't run the compliance agent alone against known cases, you can't tell which part is failing.
Give workers narrow jobs. A vague specialist produces vague output that the orchestrator has to guess about.

The Reliability Trade-Off

Here's some simple arithmetic that tends to land with technical leads.

Suppose each step in a sequential chain is right 95% of the time. That sounds high. Chain five of them and the end-to-end success rate is roughly 77%. Chain ten and it's about 60%. These are illustrative numbers, not measurements, but the shape is real: errors multiply down a chain.

Multi-agent systems add handoffs, and every handoff is another step that can go wrong. In a sequential pipeline (research, then draft, then check), you've built exactly the kind of chain where this bites.

Parallel designs behave differently. If three research agents explore separate questions independently and one failure doesn't poison the others, the math is kinder, and you can tolerate a failed branch.

There's a second reliability effect that works in multi-agent's favor, and it's the strongest argument for the pattern: independent review. An agent checking its own draft tends to share the blind spots that produced it. A separate agent, with its own instructions, its own context, and ideally a different model or a stricter prompt, can catch problems the author won't. This is the same reason companies don't let the person who wrote the contract be the only one who reviews it.

That's why, in the proposal example, the compliance agent is the specialist we'd most readily justify.

When a Multi-Agent System Earns Its Keep

Three situations come up repeatedly where we'd reach for multiple agents.

1. The work is genuinely parallel and breadth-first. Researching ten competitors, scanning a hundred documents, or exploring several independent angles of a question. Workers can run at the same time, don't depend on each other, and each handles a small context. This is the pattern Anthropic's research system uses, and it's where multi-agent has the clearest edge.

2. One context can't hold the job. If the task needs more information than fits in a single context window, or the agent degrades as its context fills with noise, splitting into specialists with focused contexts can help. Check that this is actually your bottleneck first. Better retrieval or summarization often fixes it more cheaply.

3. You need independent checks or separation of duties. Compliance review, fact-checking, security validation, anything where the checker shouldn't be the same entity as the maker. Here you're not adding agents for capability. You're adding them for accountability.

Notice what's missing from that list: "it feels more organized," and "it mirrors our org chart." Those are real reasons people build multi-agent systems, and they're poor ones.

When It's Overkill

Here's the part where we try to talk you out of it.

Multi-agent is probably overkill when:

The task is sequential and each step depends tightly on the last. Splitting it just adds handoffs.
The agents would all need the same large context. You'd be paying to copy it around, and details will get lost.
The tools fit comfortably in one agent. If one agent can handle eight well-described tools, you don't need three agents with the tools divided up.
A fixed workflow would do. If the steps are always the same, a coded sequence is cheaper, faster, and easier to debug than an agent deciding the order every time.
Volume is high and value per task is low. The token multiplier will eat your margin.
You haven't tried the simple version. If you haven't seen a single, well-prompted agent fail in a specific way, you don't know what you'd be fixing.
Your team is small. Every extra agent is another thing to prompt, test, monitor, and maintain.

There's also an architecture trap: multi-agent systems where several agents write to the same shared state (the same document, the same database record). They tend to conflict, overwrite, or duplicate each other. Parallel reading is easy. Parallel writing is where things get messy.

A Simple Decision Framework

Work through these questions in order. Stop at the first "no" that applies.

1. Does a single agent with good tools, a clear prompt,
   and decent retrieval fail on your real test cases?
      NO  -> Stay with a single agent.
      YES -> continue

2. Is the failure caused by one of these?
      a) work that should run in parallel
      b) context overload
      c) need for independent review
      NONE -> Fix the prompt, tools, or retrieval first.
      ONE OR MORE -> continue

3. Would a fixed workflow (code-defined steps) solve it?
      YES -> Use a workflow. Cheaper and easier to debug.
      NO  -> continue

4. Is the task valuable enough to justify
   several times the token cost?
      NO  -> Reconsider scope, or accept the single-agent limits.
      YES -> continue

5. Can you trace, test, and monitor each agent separately?
      NO  -> Build that capability first.
      YES -> Multi-agent is justified. Start with the smallest
             version: one orchestrator plus ONE specialist.

Applied to the proposal example, the honest result for most teams is this: start with a single agent that researches and drafts, then add a separate compliance-check agent. Not three specialists and an orchestrator. One extra agent, added because independent review is the specific need.

Notice the two intermediate options. Before you build a full orchestrator, consider:

A single agent that calls another agent as a tool. The "compliance checker" is just a function the main agent invokes. You get independence without a complex coordinator.
A workflow with an LLM review step. Draft, then a separate model call checks against a rules list, then a human approves.

Both give you most of the benefit at a fraction of the complexity.

Mistakes We See Teams Make

Designing the org chart, not the system. Agents named "Manager," "Analyst," and "Reviewer" because it looks sensible. Split by what the model needs, such as context, tools, and independence, not by job titles.

Giving every agent every tool. Then you've got multiple generalists, which is the worst of both worlds.

Letting agents negotiate freely. Open-ended agent-to-agent conversations are hard to bound and expensive. Prefer structured handoffs with defined formats.

No budget limits. Without caps on steps, tokens, and spawned sub-agents, a loop can run up a bill quickly.

Using the same model and prompt for the checker. A reviewer that thinks exactly like the author catches little. Vary the instructions, and where it makes sense, the model.

No evaluation set. If you can't measure whether the multi-agent version beats the single-agent one on your own cases, you're guessing. Build a test set first and compare both designs.

Skipping human approval for risky actions. More agents doesn't mean less need for a person to approve payments, external messages, or legal commitments.

Implementation Checklist

Before you decide

Build the single-agent version first
Write 20 to 50 realistic test cases, including hard ones
Record where and how the single agent fails
Estimate cost per task for each design at your real volume

If you go multi-agent

Define each agent's single job, tools, and boundaries
Use structured handoffs, not free-form summaries
Add one shared trace ID across all agents
Cap steps, tokens, and the number of sub-agents
Make each agent testable on its own
Use a distinct prompt (and where useful, model) for review agents

Before launch

Compare single vs. multi-agent results on your test set
Confirm human approval on sensitive actions
Test failure modes: tool outages, conflicting results, missing data
Set up cost, latency, and quality monitoring

After launch

Review traces weekly at first
Re-run your test set whenever you change a model or prompt
Remove agents that aren't earning their cost

Frequently Asked Questions

What is the difference between a single agent and a multi-agent system?

A single agent is one LLM-driven loop with tools that handles a task end to end. A multi-agent system splits the work across several agents, often an orchestrator that delegates to specialists, each with its own instructions, tools, and context.

Is a multi-agent system better than a single agent?

Not by default. It performs better on some tasks, particularly broad, parallelizable research or work needing independent review, but it costs more, is harder to debug, and adds failure points. Many tasks are served better by a single agent or a fixed workflow.

How much more does a multi-agent system cost?

It varies by design and task. Anthropic reported roughly 15 times the tokens of a chat for its multi-agent research system, compared with about 4 times for a single agent. That's one system's measurement, so estimate your own costs with real test cases.

What is an orchestrator agent?

An orchestrator is the coordinating agent in a multi-agent system. It interprets the task, breaks it into subtasks, assigns them to specialist agents, and combines their results into a final output.

When should I use a workflow instead of an agent?

When the steps are known and mostly fixed. A coded workflow with LLM calls at specific steps is cheaper, faster, more predictable, and easier to test than an agent deciding its own path each time.

How do you debug a multi-agent system?

Use a shared trace ID across every agent, log the handoffs between them, keep handoff formats structured, and make each agent testable in isolation. Most multi-agent bugs live in the handoffs, not inside a single agent.

Should the compliance check be a separate agent?

Often yes. Review benefits from independence, and a separate agent with its own instructions can catch issues an author-agent overlooks. It can be a lightweight addition, such as a second agent called as a tool, rather than a full multi-agent architecture. For regulated work, keep a human approval step in the loop as well.

Can I start with a single agent and move to multi-agent later?

Yes, and it's usually the best path. If you keep tools well-defined and logic modular from the start, you can split out a specialist later, such as a reviewer, without rebuilding the system.

The Real Takeaway

The question isn't "single or multi?" It's "what is failing, and what's the cheapest fix?"

Sometimes it's a better prompt. Sometimes it's retrieval. Sometimes it's a fixed workflow. And sometimes it really is a second agent, because you need an independent reviewer, or the work genuinely splits into parallel threads. Those cases exist, and in them, multi-agent is the right call.

The trade-offs are honest ones: more cost, harder debugging, more places for errors to hide. Pay them when the benefit is concrete, and skip them when it's just tidy-looking architecture.

If you're deciding how to structure an agent system and want a second opinion before committing, we're glad to look at your use case. At O2Devs we build production agents and multi-agent systems, and we'll tell you plainly when you don't need the second one. Get in touch and we'll go through it with you.