← boanderson.com

Everything I Know About Managing AI Agents I Learned Managing People

How I ship production iOS apps around a day job, by managing AI like an organization instead of a tool.

Bo Anderson · July 2026 · 9 min read

I'm a marketing executive. Fifteen years of demand generation, product marketing, and P&L ownership. No computer science degree. I have a day job I take seriously and a life outside of it.

I also have two iOS apps live on the App Store, three more products in flight, and a self-hosted system that builds and verifies software while I'm in meetings. Everything was built at night, on my own accounts and my own server, in blocks of about four hours. The hours I used to give to television.

I want to be careful about what this is a story of. I got lucky on timing: the models got good at exactly the moment I started trying. What I brought was a set of management habits that transferred to a place I didn't expect them to. This is a walkthrough of how it actually works, including the bugs, because the bugs are where the system came from.

I started by doing it wrong

When I first started building with AI coding agents, I did what everyone does: I supervised. I sat with the session. I watched it work. I read diffs I was only partially qualified to evaluate. I answered its questions in real time and caught its mistakes when I was lucky.

It worked, in the sense that software got built. But it made me the bottleneck, and worse, it made me the quality bar. I'm a marketing guy. A system where the least technical person in the room personally inspects all the work is a bad org design. I would never run a team that way.

That reframe was the unlock, at least for me. I've spent fifteen years trying to get good work out of people who are better than me at their jobs. Executives do it with structure: clear briefs, acceptance criteria, independent review, escalation paths, and a rule that "trust me, it's done" is not a status. So I stopped trying to be a better supervisor and started building the management infrastructure instead.

I call it Relay.

The org chart, in software

Relay is a daemon on a Linux VM I rent, wired to a Postgres database. It runs the coding agents; I run Relay. Six project codebases live under it today: the shipped apps, the ones in flight, and Relay itself, which is built through its own pipeline.

Work enters as a brief, not a vibe. A brief has a goal, acceptance criteria, non-goals, and an expected size. Before any agent sees it, a linter scores it. The gate is deliberately fail-open, and most briefs queue immediately, clean or flagged. But two things stop a brief cold: it contradicts a standing design decision, or it involves destructive or paid work with any property left uncertain. Sloppy delegation used to produce sloppy sessions; now the system refuses my sloppiness before it costs me a night.

Agents work autonomously from there. No babysitting. But no agent grades its own homework. When a build session finishes, a separate session starts with no memory of the first, in read-only mode, and audits the work against the brief. It ends by filing a structured verdict: pass or fail, with the gaps named. Builders build; verifiers verify; the separation is structural rather than aspirational.

Institutional memory lives in design records: principles, decisions, and how-tos in the database, not in anyone's chat history. Every working session gets a fresh context pack regenerated from live system state before it starts. One of Relay's standing principles says it plainly: prompt text and chat memory are never the authority. Chats are ephemeral; the org's knowledge shouldn't be.

And status, the part I care most about, is an evidence projection. The dashboard doesn't display what an agent claimed. It displays what the evidence supports, and when proof is missing it renders unknown. Not "done." Not green. A related rule I've come to rely on: an empty work lane is truthful. The system isn't allowed to invent work to look busy.

My own attention is rationed by contract. I don't watch sessions; I don't even see them. I track features and releases, and the only thing that's allowed to interrupt me is a typed "Needs You" packet describing a specific human action: a decision, a device test, an approval. Failures alone don't page me; the system recovers or escalates with a full context block I can paste anywhere. Four hours a night only works if the four hours are spent deciding, not monitoring.

What the machine is not allowed to do

Autonomy without boundaries is how you wake up to a disaster, so the boundaries are hard-coded, not customary.

Consent gates. Anything destructive or irreversible, anything involving real money, and anything public-facing (releases, external posts, payments) stops and waits for me. Everything else can run at 3 a.m. without asking.

Sanitized visibility. The web surface I check from my phone shows work state, timestamps, and outcomes. Not raw commands, logs, or secrets. If my browser session were compromised tomorrow, the attacker would mostly learn that a feature is "Verifying."

Isolation. Verification runs against a staging environment that structurally mirrors production: schema and functions, never data. The test harness rejects any host that isn't the approved staging instance before a browser ever launches. Production migrations gate on me, every time.

A kill switch. One durable flag reroutes every model call in the system to a single backend. When you run an autonomous system, it seems wise to build the master breaker first and hope you never need it.

The bugs that wrote the rules

Every invariant above is a scar. Three worth showing.

The silent wipe. Early on, re-running my app's onboarding flow silently destroyed the user's existing data. A fresh start, helpfully clearing the table. Precisely wrong. The fix wasn't just a patch; it became standing law: no flow may silently destroy user data. Destructive paths must archive, confirm with specifics, and offer explicit restoration. That principle is now checked against every brief that touches those surfaces.

"Done" is not "passed." A release-gate worker finished its session and reported done while the test it existed to run sat untested. A downstream step saw done, considered its dependency satisfied, and ran anyway. Its done then concealed the failure one layer deeper. The lesson I keep relearning: an agent completing its task is not evidence the work is correct. Release gates now depend on structured proof: the exact test, the exact build fingerprint, a terminal passing verdict. A predecessor merely finishing satisfies nothing.

The zombie session. An agent's wrapper process died without filing a report. The row said "running"; nothing was. The naive fix (wait out the clock, requeue) risks re-executing work that may have partially landed. The fix I landed on reconciles process identity and git ground truth within a bounded grace window. No process and no landed commit, safely requeue. A commit landed, hold for human review. Never requeue a live process, never duplicate an external mutation. Distributed-systems hygiene, learned the slow way.

I fix bugs like these most weeks. Some weeks I'm debugging the manager, not the code. The system gets more trustworthy only because nothing gets to claim trust without evidence, including the parts I designed.

The front office is a chat window

One choice was deliberate from the start, and it shapes everything else: Relay has no dedicated front end. The front end is whichever chat app is in my pocket. I built the system so that Claude or ChatGPT is how I operate it, on purpose, because a chat window is the one interface I'm guaranteed to have on any device, anywhere.

The mechanics are less mystical than they sound. Through a small MCP interface, the models can read Relay's live state: a handful of read-only, feature-scoped tools behind bearer auth, because giving agents a bounded API felt safer than handing over keys to the filing cabinet. So when I open a chat about a feature, the conversation is grounded in what's actually true right now, not in what an old thread claims. The steering happens there too. When I need to see an interface before I commit to building it, I ask Claude to mock it up with its built-in visualizer, iterate on the mockup right there in chat, and write the brief against the thing I can already see. Specs get sharpened the same way, by arguing with a model until the acceptance criteria survive.

Two things fall out of this. I can run the whole operation from anywhere, on anything, because the company lives behind an app I already carry. And the front end improves without me touching it: better models, new tools, richer visualizers arrive the day the labs ship them, with no integration work on my end. I don't maintain a design tool or a research tool; I'm borrowing them from companies that are much better at building them than I am.

Where it's going

The current build-out, Relay Next, is an attempt at a better operator surface: a Home view of live work derived from evidence, a release board where features carry their own test contracts and proof, and backlog planning that can't quietly drift from readiness truth. I half-jokingly call it the CEO console, though the CEO in question manages zero people.

One more piece I find interesting as architecture, with the caveat that a real platform engineer might find it obvious: the setup is model-agnostic on both ends. The chat front end can be either lab's app, and the coding-agent labor underneath is interchangeable across backends; I can switch providers with a flag. The governance layer in the middle stays constant. That's at least how I was taught organizations are supposed to work: the structure outlasts any individual seat.

Why a marketing guy could build this

I don't know how far my case generalizes. These are small consumer apps, not systems anyone's life depends on. I'm the only stakeholder, which makes governance easier than it would be for a real team. And the models improved underneath me the whole time, which flatters every decision I made.

With those caveats, here's what I can say honestly. The hard part was never the code; the models write the code. The hard part was the stuff my career happened to train: writing a brief precise enough to delegate against, defining "done" as evidence rather than assertion, designing review that doesn't depend on the reviewer being smarter than the worker, deciding which mistakes are reversible, and building escalation paths so problems find you. That's management. For me, it transferred.

For twenty years, the constraint on building software was engineering hours. If that constraint really is dissolving, then the next one might be management clarity: the ability to specify, verify, and govern work you can't personally perform. If that's even half right, a lot of operators are sitting on more leverage than they realize.

Last night the verification came back green while I slept. I read the evidence over coffee and approved the release myself. That last part never gets delegated.

BO ANDERSON · @BOANDERSON · HOME