What broke when I ran 20 coding agents at 229B tokens a month

One of my coding agents once ran for 4.5 days, used 40.8 million tokens and made zero commits. Another reported CI green after running 28 of 308 tests. My worst outage so far ran for hours because a change reached production while I was asleep and my phone was dead.

Each of those came down to how I was running the agents. I'm one developer working through 10 to 20 of them in parallel, about 229 billion tokens a month, and what fixed most of it was to stop acting like a junior engineer who types faster with help and start acting like a manager who also codes. With ten agents you're the head of a ten-person team, whether you act like it or not.

The agent is only as good as the repo it lands in

The clearest difference in quality I've seen came from the codebase. On a fresh repo with a good foundation (domain-driven design, real integration tests against a real database, a QA pipeline with a gate, and an architecture that fit the project) the agents were fast and the output needed little correction. On an inherited codebase with years of issues, the same models and the same workflow produced noticeably worse work. They copied the patterns they found, and those patterns were the problem.

So should you start from scratch? If the codebase is small, I'd say yes, clearly. Rewriting a few thousand lines with agents is cheap now, and you get a baseline they can learn from.

If the codebase is big, in production and somehow working, I wouldn't. I migrate gradually instead: refactor one area into the shape I want, delete what's bad, and let that area become the reference the agents imitate. That only works if you know your codebase well enough to say which parts are the good example and which parts should go. The agents can't make that call for you, because to them every file in the repo looks equally authoritative.

Frontier models plan and review, cheaper models write the code

A lot of people still use the top model for everything, Fable or Opus 5.5 on every task. It gets the work done, but getting it done isn't the goal. The goal is a balance between effectiveness and efficiency.

My setup is planner, executor, reviewer. A frontier model writes the plan and reviews the result, and the execution in between goes to cheaper models like GLM, Grok or DeepSeek, whichever is good enough for the task. Writing code from a clear plan needs the least judgment and burns the most tokens, so that's where a cheaper model pays off. In my own runs, Claude spend per task dropped to about half of what it was when Claude did everything.

Subscriptions work the same way. I had a $20 Codex plan I barely used, so now Claude hands research to Codex whenever it needs to look something up. Research is mostly reading and searching, and a plan sitting idle is the cheapest place to do it.

The split only holds up when the plan is good. A cheap executor follows a vague plan just as faithfully as a precise one, so most of the quality now comes from how I write the brief.

A brief is a contract, and the tests come first

My briefs used to describe what I wanted. Now they're contracts: the interfaces and invariants, plus acceptance tests that prove the feature works. The tests get written before any implementation and have to fail against the current code. That keeps the executor's job narrow and checkable, and a QA gate decides whether the work passes. The agent's own report doesn't count.

For anything with a UI, I build the feature as a mockup first, usually a plain HTML file. Done then means something concrete: the UI is pixel identical to the mockup, and feature X behaves the way the mockup shows. The agents get a target to compare against instead of a description to interpret.

Where this falls down is testing, because AI is bad at tests and worse at QA. An agent will tell you it's done, and when you check, it is, in the sense that the basic path works. The checks it ran often couldn't fail. Besides the CI run that tested 28 of 308, I've seen a piped tail turn a failing test run into exit code 0, and a parity check report zero failures while only 9 of 49 screens matched the mockup.

Then come the edge cases, which are less edge than you think. When we shipped an AI product to real users, people invoked it in ways I would never have thought of. Lesson learned: for the next feature, the brief told the agent to test at least a fixed number of edge cases and to name each one. That worked noticeably better. There's always someone with a very special requirement nobody predicted, and that's normal, but the obvious-in-hindsight ones stopped reaching users.

Goals have to be small enough to end

The 4.5-day run had a finish line that needed a step only I could do. Nothing was technically broken: the agent kept doing the cheapest thing that looked like progress, which was producing more evidence about the work instead of doing the work.

Now every goal is scoped and small, names the deliverable, and can actually end without me.

More rules made the agents follow fewer of them

The more context you give an agent, the more it starts to bend your rules or ignore them. One shared rule file of mine grew from 434 to 1,554 lines in a single day, as every fix added another paragraph, and past a certain length the agents stopped following it reliably. What still works for me is a short file, with the rules that matter enforced by a hook or a test so the agent can't talk its way past them.

That's also where off-the-shelf workflows hit their limit for me. Superpowers, pstack and gsd are great, and I learned a lot from them. But they're built around someone else's way of working, and the moment yours differs you're fighting the framework. One feature of mine ran for over a week and still wasn't done, largely because of the process around it.

You know your workflow better than any framework author does, so write it down as a skill and adapt the popular ones to it. Mine became keryx, which runs the contract, tests-first and QA-gate loop from above and borrows whatever fit from those frameworks. Compared with my old setup, the share of runs where an agent claimed done and wasn't dropped from 36% to 11%.

CI wasn't built for ten agents pushing at once

CI/CD is a pain with parallel agents. Current systems assume a few humans pushing a few times a day. Ten agents push constantly, each one waits for its own run, and superseded pushes keep running anyway. A solo developer with a $1,000 CI bill sounds absurd, and it's avoidable.

I got CI time per agent from about 60 minutes to under 10, and the cost to around $120 a month. Most of that came from splitting test suites into parallel jobs with one database per worker, and from moving off large runners that spent longer waiting in a queue than running.

AI can't do CI on its own, though. Agents will happily make a pipeline green by skipping tests or making a job non-blocking. You have to steer: decide what the pipeline must prove, and check that the green run actually ran what it claims.

My MacBook became the bottleneck

With 10 to 20 agents in parallel, at some point the limit stopped being the model and became my MacBook Pro. Three agents running heavy checks at once pushed my 32 GB machine to about 37 GB of swap. Add the browsers test tools leave open and the dev servers nobody stopped, and the laptop turns into a piece of metal as hot as the sun.

If your agents run 24/7, move them off your laptop, to a homelab or a VPS with 32 GB of RAM. Mine now run on a server, and the Mac is where I plan and review. Coordinating agents across machines turned into its own problem, which is why I built Morndo and open-sourced it along with keryx. Both started as fixes for my own bottlenecks.

No production merges while I sleep

The outage from the top cost more than any bad code an agent has written for me. PagerDuty went off, the escalation went off, and my phone was dead, so a few hours passed before anyone noticed.

Now nothing merges to production while I'm asleep. Agents prepare everything overnight (the branch, the tests, the review, the rollout notes) and the merge happens when I'm awake to see the alert.

I still read every SQL query that touches production

There's one place where I don't delegate, and that's anything system critical. I read every SQL query before it touches a production database.

If you decide to take that risk and let an agent handle database operations, at least make sure your backup and rollback actually work, and test them before you need them. We had a production incident of exactly that kind. Because the backup and rollback had been proven beforehand, we lost about 15 minutes of data.

Your logs are an asset, if you keep them in order

Collect your data and keep every transcript and cost figure. I believe data is one of the few real assets in this era, and it's how I know which model did what and which changes actually helped. Claude Code deletes transcripts after 30 days by default. When I counted 225 billion tokens last month, most of my history before September was already gone.

Storing it isn't enough, though. The data has to be structured and organized, or it's just a bigger pile to search. Mine goes into a ledger that dedups across machines, so any number I quote is one I can rebuild.

If someone starting with coding agents today asked me for one sentence, I'd tell them to build systems, not products.