On this page
An AI agent has more in common with a product than a project, and that changes how you build it, staff it and plan for it. The first piece in Built to Run, on AI work that keeps running.
Our phone agent, Clarke, started answering InTech's line a few months ago. By the usual definition of a software project, that was the finish line. It answered calls, sorted them into five kinds, and emailed us a record of every one.
Then we read the records. Clarke told one caller someone would follow up "first thing Monday," a time nobody here had agreed to. It promised another a callback from a person who doesn't exist. It logged a call that had nothing to do with sales as a new prospect. A caller asked for an appointment, and Clarke had no way to book one.
Version two shipped about two months later. A later update let it book discovery calls inside windows a person has opened. Clarke is still changing, and I expect it to keep changing for as long as it answers the phone.
That's the pattern behind this series. Most companies buy AI work the way they've always bought software work, as a project with a scope, a delivery date and a sign-off. Then they're surprised when the thing they bought keeps needing attention after the sign-off.
The work was a product all along, and they bought it as a project.
A project gets delivered
A project has a shape everybody understands. You agree on what it should do, build to that, test against it, and when it passes, the client signs acceptance and the final invoice goes out. The team moves on to the next thing.
That shape works when the thing you built behaves the same way next year as it did on the day it shipped. A data migration is a project. So is a report that runs every Monday, or an integration between two systems that don't change much.
They'll need upkeep, but their job on day 300 is the same job they had on day one.
Done, for a project, means the work stops. That's what makes it easy to scope, price and close.
A product keeps evolving
A product has users, a backlog and releases. Nobody expects it to be finished. People expect it to get better, and they expect someone to own that.
An agent behaves like a product from the first week. The team that wrote Clarke's first instructions knew what InTech would and wouldn't say on the phone. Callers didn't, and within a few weeks they'd found the gaps.
Its behavior gets shaped by the people who talk to it, and those people don't read the requirements.
Three things keep moving after launch. Callers keep asking for things nobody planned for. The business keeps changing what it's willing to promise. And the models underneath keep changing too. Clarke's second version moved to a newer model and a new voice, and that release was tested against fourteen simulated callers before it took a live call.
Put Clarke's short history in a row and it reads like product release notes: a first version, a second version built from what callers actually did, and a booking feature. None of that was in a scope document. All of it came from watching the thing work.
What happens when you buy a product as a project
Clarke is our own agent, so nobody had to sign anything to fix it. Picture the same build sold to a client as a project. The statement of work would have said answer calls, route five call types, send a record. All of that worked on day one. The client signs acceptance and the project closes.
From there, one of two things usually happens.
The first is the change-order path. Every correction goes through a scope conversation, an estimate and a signature. Each fix is small, so the paperwork feels out of proportion to it, and clients stop asking. A while back Chuck ran the exit test on our own systems and found that friction, more than severity, decided what got fixed. Change orders are friction by design.
The second is the goodwill path. The firm fixes things for free to keep the relationship warm. That works for a month or two. Then the engagement is closed in the firm's books, the engineer who built it is staffed somewhere else, and the fixes get slower until they stop.
Either way the agent drifts. It keeps making the promise nobody approved, because reading the transcripts isn't anybody's job anymore. Earlier this year I asked readers to name who fixes a system when nobody is billing.
When a product is bought as a project, the answer eventually is nobody.
Think about it as a product from the start
Once you think of an agent as a product, four things follow. We learned each of them on Clarke.
The first is an owner who knows the business. Look at what changed in Clarke's second version. The follow-up window became one business day. Clarke stopped offering live transfers and started offering callbacks, because nobody staffs our phone line. Vendor calls stopped landing in the sales pipeline. Every one of those was a decision about the business, and whoever made it had to know how we answer the phone, who follows up and what we're willing to commit to. That's why a product needs an engineer dedicated to it, one who knows the business as well as the code.
The second is a backlog that comes from the record. For Clarke that's call transcripts. For an agent that drafts documents, it's what the agent wrote and what a person changed before it went out.
Somebody has to read it on a schedule, because the record is where the next version comes from.
The third is a release discipline. Clarke's prompts, knowledge base and test cases live in a repository, so every change is traced and every release is tested before it reaches a caller. A product that changes every few weeks without that discipline turns into something nobody can explain.
The fourth is a way out. A while back Chuck and I wrote about the exit test: could a client walk away quickly with everything that's theirs? For a product that keeps changing, that means the accounts, the repository and the tests sit in your name from the start, and you can see what the agent costs to run without anyone in the middle.
The budget follows from those four. A product needs money for the first version and then for the people and the discipline that keep it right. Budget it as a single project with an end date and you've paid for the first version and left everything after it to change orders or goodwill.
What I'm still working out
The first open question is where the line sits. Not everything we build with AI is a product. Some of it really is a project, and treating a one-time data cleanup like a product just adds a monthly bill to something that's finished. I can tell the difference after a few weeks of watching the thing run. I'd like to be able to tell before we start, and I don't have a clean test for that yet.
The second is the owner on the client's side. A dedicated engineer can learn a business well, but the decisions in Clarke's second version came from the people who run ours. On a client's agent, someone inside the company has to own what it may promise, and most companies haven't named that person when the work starts. I haven't figured out how to make that part of the first conversation instead of something we discover a couple of months in.
Land the plane
If you're planning AI work, whether you build it yourself or hire someone, three questions will tell you whether you're thinking about it as a project or a product.
Who owns it the month after it ships? If the answer is a support queue, you're still thinking project. A product has a named owner who knows your business.
Where does the next version come from? A product's backlog comes from someone reading what the agent actually did. If nobody is reading, it won't improve except by accident.
What do you own if the people who built it go away? You should be able to name the accounts, the repository and the tests, and all of them should already be yours.
Tell me in the comments about something you bought as a project that turned out to be a product, and what happened in the month after it went live.
Next week I'll walk through what one month of running an agent actually involved, from the first transcript to the release at the end of it.
