Why we measured it
More people now run several agents at once, and more of what agents do happens on live websites. Two questions follow quickly. Do more agents finish sooner? And what stops two of them clicking the same button, or one acting on another's unverified claim?
We build M9R, software that lets agents from different vendors act on the web together under a person's control. We wanted numbers before making claims, including numbers that don't flatter the idea.
How we tested
A deterministic local website with two tasks, each with a stored correct answer and a scorer. Search has 24 product pages and exactly one target certification code, plus decoys that differ by a single character. Trip is a dependent chain where each lookup needs the previous result.
The agents were real Claude Code sessions (Sonnet) and, in a separate set, real Codex sessions, started through their official command-line tools on the operator's own subscription logins. Every run refused to start if an API key was set. Built-in browsing and shell tools were switched off, so the only way to reach the site was M9R's browser tools (open, read, click, type) and its message tools, which drive a real headless Chrome through a local broker.
Each task ran under three conditions, same seed each time:
- Solo
- One agent does everything.
- Parallel
- Three agents work independently and report at the end.
- Coordinated
- Three agents share findings while they work.
We recorded time until the correct answer was submitted, correctness, tokens, turns, page loads and messages. Dollar figures are API-equivalent estimates from Claude Code's own output, not what a subscription charges.
Search, wide work
6 seeds per condition, seconds
Trip, short dependent chain
6 seeds per condition, seconds
Search
median across 6 seeds, thousands of tokens; whiskers show min to max
Trip
median across 6 seeds, thousands of tokens; whiskers show min to max
| Condition | Time to answer | Tokens | Est. cost | Page loads | Messages | Correct |
|---|---|---|---|---|---|---|
| Search: wide work | ||||||
| Solo | 66.2 s | 386,896 | $0.204 | 25 | 0 | 6 / 6 |
| Parallel | 41.2 s | 362,934 | $0.272 | 31 | 2 | 6 / 6 |
| Coordinated | 42.6 s | 600,684 | $0.327 | 24 | 2 | 6 / 6 |
| Trip: short dependent chain | ||||||
| Solo | 28.4 s | 147,149 | $0.087 | 6 | 0 | 6 / 6 |
| Parallel | 36.0 s | 286,849 | $0.232 | 14 | 2 | 6 / 6 |
| Coordinated | 33.8 s | 294,347 | $0.211 | 10 | 5 | 6 / 6 |
What we learned
Parallel agents help when the work is wide
Three agents beat one by about 38% on search, at roughly a third more cost. The task splits cleanly, so the extra agents shorten the wall clock.
They don't help on a short chain
On the trip task, one agent was fastest and cheapest. Later steps can't start until earlier answers exist, so extra agents mostly add overhead: about twice the tokens and 2.7 times the cost, for slower answers.
Coordination didn't buy speed
On search, coordinated runs were no faster than plain parallel (42.6 s against 41.2 s) and used far more tokens. On the trip task they were about two seconds faster and loaded fewer pages, a gap too small to lean on with six seeds. We suspect much of the search overhead came from agents checking their inbox and waiting, but we haven't isolated the cause.
Agents trusted each other without checking
In every mixed run, the agent that submitted the final answer said, in its own words, that it had taken a teammate's report without verifying it.
“…a3 reported it as FOUND, and I submitted it without checking its detail pag…”Seed 1 · submitting agent
“…It comes from a3's ‘FOUND P-34’ message, which I did not check on the detail…”Seed 2 · submitting agent
“…a3 reported it, and I submitted it without checking its code mysel…”Seed 3 · submitting agent
“…a3 reported it as the match, so I stopped my own checks and submitted it. The an…”Seed 4 · submitting agent
Quotes are verbatim, cut at 100 characters by our run logger. All four submitted answers were correct. With a decoy, a hostile page or a mistaken teammate, the same habit turns a small error into a submitted one.
We built a verified-findings rule for this: a claim isn't reliable until an agent other than the reporter has confirmed it, and any rejection blocks it. It has tests. We haven't yet measured what it changes in real runs.
Cross-vendor coordination works, with setup
In four mixed runs (Claude, Codex, Claude) on the search task, all were correct, in 33 to 48 seconds, against a Claude-only median of about 43 seconds. The Codex agent read its teammate's finding from the shared inbox and stopped. Getting Codex to use the tools took three changes: allowing its code gateway for tool calls, pre-approving the tool server for non-interactive runs, and letting it start in a folder that isn't a repository. These are details of Codex CLI 0.153.4 and will change.
Time to the correct answer, Claude + Codex + Claude
4 seeds, seconds; the marker shows the Claude-only coordinated median (42.6 s)
What we built to keep it safe
Speed benchmarks don't show the problems of several agents sharing one real browser. Three mechanisms we added:
- Claims. Typing or clicking in a tab takes a short-lived claim. A second agent trying to act on the same tab is refused with a clear message, and the page shows a lock and a blocked tag. Reading is never blocked.
- Scoped permission for someone else's agent. An agent belonging to another person acts only under a grant that names one site, optionally one path, the allowed actions, an expiry and a use limit. The browser's owner approves it and can revoke it.
- A tamper-evident log of every grant and every guest action, metadata only. It detects accidental edits. It is not proof against someone who can rewrite the file, and we say so.
Limits of this study
- Small samples. Six seeds per condition for Claude and four for the mixed team. Medians were stable, ranges overlap in places, and none of this is statistical proof.
- One site family, two task shapes. Real websites are messier.
- One Claude model tier. Other models may behave differently.
- Codex's contribution wasn't measured. We know it read and reacted to messages, not how much it searched.
- Costs are estimates. API-equivalent for Claude only; mixed-run costs undercount.
- Our harness, our product. We wrote the site, the runner and the tools. Weigh the results accordingly.
What this means
If you hoped more agents means faster, the evidence says: sometimes, and mostly when the work is wide. The stronger reason to put agents in one browser is what a single agent can't offer. Agents from different vendors, and eventually different people, act on the same live pages while a person watches, sees who is doing what, and approves the one step that matters. That needs claims, scoped permission, a record and honest limits. It is a safety and visibility problem first, and speed is a bonus.
What's next
- Delivering agent-to-agent messages without polling, to see how much of the coordination cost goes away.
- Testing the verified-findings rule against a decoy that a teammate reports wrongly.
- More vendors through the same tools, claiming only the ones we've actually run.
- Publishing the results of a scripted attack suite against the extension and broker.