On this page
How our phone agent went from version one to version two, and why most of what changed was about the promises it was allowed to make.
Last week I wrote about what our phone agent decides without us. I said the safeguard at that rung is a short written list of decisions plus a record someone reads soon after the call. This week is the part I promised: how the first version of that agent became the second, and what moved us from one to the other.
When people hear "version two" of an AI agent, they assume the model got better. Ours did get a newer model and a newer voice in the second release.
Those were the least interesting lines in the change list.
Most of the rest came from two places: reading what the first version said to real callers, and lining it up with the Clarke on our website. And most of what we changed was about promises: what the agent was allowed to commit the company to, and what it had been committing us to without anyone deciding it should.
What version one was
The first version went live the first weekend of August. I didn't build it by hand. I scoped it in a long working session, wrote the decisions into a handoff document, and gave that document to a development agent to build.
The decisions were deliberately plain. The agent runs on ElevenLabs (opens in a new tab) end to end, so one platform covers the agent, the voice and the phone line. It uses a stock professional voice, a male voice named Clarke. It isn't my voice clone.
Behind it, a small service takes the record of each finished call and sends it three places: an email, a chat message, and the same lead pipeline our website intake feeds. A phone lead gets evaluated exactly the way a web lead does.
There was an earlier round. Last winter we tried a number of different tools, and Vapi, n8n and Teams were the ones we really kicked the tires on. That was testing and never went live, so the story starts in August.
Every call produces an email. It has who called, the number, how long the call ran, the category Clarke chose, the urgency, a summary and the full transcript. That email is the record I wrote about last week.
What the transcripts showed
The first problem showed up on one of the first calls, which I made myself. I called in on a Sunday as a made-up founder with a transportation app and said I wanted to get started as soon as possible. Clarke was friendly, quick and on topic. Then it closed the call by telling me the team would follow up "first thing Monday."
Nobody had decided that. No one at InTech had agreed to Monday morning for a weekend caller. On the same call I told it to use the number I was calling from, and it asked me for my number again.
The agent picked a sensible-sounding time and put the company's name behind it.
The second problem was a real caller in September. They asked for someone by a name that shared my last name but wasn't mine. Clarke didn't ask who they meant. It told them "Kevin" would get the message and follow up. There's no Kevin at InTech. The call was about board opportunities with a local business organization, and the relay filed it in our lead pipeline as a new prospect. So the agent promised a callback from someone who doesn't exist, and a call about joining a board landed in the sales pipeline next to actual prospects.
The third was a twenty-three second call. The caller said one word... "appointment." Clarke started asking what kind of appointment, talked over the caller, apologized, and the call ended. Version one had no way to put anything on a calendar, so the most direct request a front door can get had nowhere to go.
The fourth pattern was less dramatic and more constant. Many calls are a few seconds of silence followed by a hang-up, and each one still produced an alert.
Every one of these showed up in the call emails, and none of them would have shown up in a count of calls answered.
What changed on October 2
Version two shipped on October 2. Two things drove it. The transcripts handed us a list of corrections, and we wanted the phone Clarke to match the Clarke on our website.
Clarke stopped inventing follow-up times. The agent now makes one promise about timing, that the team will be in touch within one business day, and that promise is written into its instructions ahead of time.
It confirms the caller ID instead of asking for a number the caller just gave. When no number is stated, the relay falls back to caller ID, and it never saves one of our own numbers as a caller's callback number.
It offers a callback instead of a live transfer. We don't staff a phone system, so there isn't anybody sitting by a phone waiting to take a transferred call. We run the business by working directly with customers, and we don't spend much of our day on the phone.
It tells callers at the start that it's InTech's AI assistant and that the call may be recorded. Version one disclosed that it was an AI. It didn't mention recording.
Vendors are welcome. Clarke is polite to someone selling us something and takes the message, and the relay sends those calls to email and chat only. They no longer land in the lead pipeline. Every new lead still goes to CRAFT OS.
To line the two up, we rewrote the knowledge base the phone Clarke answers from. The release added new fields for what a caller is interested in and whether they asked for a callback, plus new criteria for grading each conversation. Before any of it reached a caller, it ran against fourteen simulated callers.
The same release moved Clarke to a newer model and a newer voice. I asked for the speaking speed to go from 1.1 to 1.2 that afternoon.
Three days later we added the thing the twenty-three second caller wanted. Clarke can now book a discovery call during the conversation. It offers open times inside the same hours our booking page uses, reads the time and the email address back, and asks before it books. I tested it that evening and got a meeting on Steve's calendar for the Wednesday. The next day the relay stopped posting a chat alert for calls where the caller never said anything.
A date the agent invents and a date the calendar offers
Last week I wrote that anything committing the company to a price, a date or a promise goes to a person. Then version two started booking meetings on its own. That looks like a contradiction, so it's worth being precise about it.
"First thing Monday" was a date the agent made up. Nobody owned it, nothing checked it, and someone on our side would have had to keep it.
A booked discovery call is a date the calendar offered. A person set those hours. The slot was open. The caller picked it and said yes after hearing it read back. The agent is choosing among commitments a person already made available, which is a different decision from creating one.
We set those booking windows on purpose. Once a caller has been validated, they can get a time on the calendar before they hang up, which cuts the cycle time and the back and forth.
That distinction runs through the whole change list. Version one improvised promises: a time, a callback from a named person, a place in the sales pipeline.
Version two only makes promises someone wrote down ahead of time.
Where the agent's definition lives
When I asked for the speaking speed change, it was made live in ElevenLabs first. Then the same change was written back into our repository, with a note saying it was mirrored so the repo stays the source of truth.
That habit is what keeps an agent like this reviewable. The prompt, the knowledge base, the data fields, the grading criteria and the simulated callers all live in version control. When something changes, there's a record of who asked for it and why.
When I want to know what Clarke was allowed to say on a given day, I can look it up.
If your agent's definition only exists in a vendor console, you can't answer that question. You also can't run the same fourteen callers against the next version and see what got worse.
What I'm still working out
Silent calls still generate an email. We stopped the chat alert, but I'm still reading a steady trickle of three-second calls with no summary. I haven't decided whether those belong in anyone's inbox.
Language is open. A caller in late October switched to Spanish partway through, and Clarke followed after telling them the line was set up in English and might not sound perfect. That was the right behavior. We've drafted Spanish wording for a Panama line, and it's waiting on a native speaker to review it before anything goes live.
The vendor rule makes the pipeline cleaner, and it puts more weight on the classification. Last week I said the mistake I watch for most closely is a real prospect getting turned away as a solicitation. Version two makes that mistake cheaper, because the caller still gets a polite answer and a message gets taken. It doesn't make it impossible.
Land the plane
The second version of an agent comes from reading what the first version said to real people and writing down the decisions it was already making on its own.
If you're running an agent that talks to customers, read a week of its transcripts before you change the model. Mark every sentence where it committed your company to something: a time, a name, a follow-up, a category that sends the caller somewhere. Then decide which of those it should be allowed to make, write them down, and run the new version against simulated callers before a real one hears it.
Here's what I'd like to hear in the comments. If you've shipped a second version of an agent, what drove it? Was it a better model, or something you read in the record of the first one?
Next week I'll close out the Trust Ladder with a version you can run on your own business in an afternoon.
