Field notesARTICLE
The First Week Goes Wrong in Three Places
It is nearly always the same three, and none of them is the model getting an answer wrong. The first week is an audit of your own rules, conducted by something that follows them exactly.
Costa5 min read
Short answer
The first week of an assistant surfaces three things: a rule the company thought it had but did not, a handover threshold set wrong in both directions, and an answer the business had never put in writing before. All three are findings about the business rather than defects in the software, and all three are fixed in the configuration.
Key facts
- Every rule the assistant follows is a line someone in the business approved, not a model preference.
- The three findings above are fixed by editing rules, without retraining anything.
- Handover thresholds are adjusted per topic rather than globally, because the two failure directions have different causes.
- The full transcript is readable from day one, which is what makes the findings visible at all.
An assistant's first week almost never fails the way people brace for. Nobody is embarrassed by a hallucinated fact; the transcripts are dull. What happens instead is that three things surface, and they are the same three nearly every time.
None of them is a software defect. All three are things the business believed about itself that turned out not to be written down anywhere.
Day two: the price two people disagreed about
A customer asks what a two-week booking costs. The assistant answers from the rule it was given. Within the hour, two people from the same company are arguing about the answer in the group chat, and neither of them is arguing with the assistant. They are arguing with each other.
It turns out the business had two prices for that case. One person had been quoting the round number, the other had been applying the weekly rate and rounding down. Both had been doing it for a year. They had never quoted the same customer, so nobody had ever seen the two sitting side by side.
The assistant did not create that. It made the two versions collide in public, once, and then somebody had to choose. The fix takes four minutes and it is the most valuable thing that happens all week.
Day four: both directions in one day
The morning failure: a customer asks whether there is parking, phrased strangely, and the assistant hands it to a person. The information was on the page. That is a handover that cost a person two minutes and made the customer wait for something a machine already knew.
The evening failure, same day, opposite direction: a customer writes we might need to rethink the dates, things have changed on our end. The assistant answers about dates. That was a cancellation with a polite face on it, and it needed a human being inside ten minutes.
Both look like the threshold being wrong, and there is a temptation to move one dial. It is two different problems. The morning one is coverage — the assistant did not have the parking answer in a form it could stand behind. The evening one is recognition, and it gets fixed by naming the pattern rather than by raising a confidence number: a hedge about rethinking or things changing from an existing customer goes to a person, whatever it is about.
Day six: the answer nobody had ever written down
Someone asks what happens if they cancel late.
The assistant tells them, accurately, from the terms. And the owner reads it and feels something go tight, because in nine years nobody in that company had ever answered that question that plainly. In a phone call it gets softened, in an email it gets postponed, and everybody had adapted to a policy that was never quite said aloud.
Nothing here is a bug. The terms are the terms. What surfaced is a decision that had been deferred for nine years: either the policy is defensible in writing to a stranger at eleven at night, or it needs changing. Both are legitimate answers, and the assistant is the first thing in the business that has ever forced the question.
The instrument and the reading
Not accuracy. An assistant that follows your rules exactly is a measuring instrument pointed at the rules, and week one is the reading.
That reframes what to look for in the transcripts. The interesting lines are not the ones where the answer is wrong. They are the ones where the answer is right and somebody in the company flinches.
What to do with the seven days
Three things, and only the first is work.
Somebody who is allowed to change rules reads every conversation that week — not a sample, all of them, which is an hour a day at the volume most businesses have. Reading without the authority to edit produces a list of grievances; reading with it produces changes by Wednesday.
Second, edits go into the rules the same day they are found, one line at a time, so that Thursday tests Wednesday's fix.
Third, resist the urge to launch somewhere safe. A quiet channel with no traffic does not de-risk the first week, it postpones it to a month when nobody is watching. Launch where the messages are, while you still have the attention to read them. Omni AI puts every channel into one queue precisely so that this is one hour and not five.
By the second week the transcripts get boring again, and that is the state you were trying to reach. What you keep from week one is not a better model. It is a price your company agrees on, a rule about the word rethink, and a cancellation policy somebody finally decided to stand behind.
Questions this raises
- Should we launch quietly on a small channel first?
- Only if that channel gets real volume. A quiet launch on a dead channel postpones the first week rather than de-risking it, and the whole value of these seven days is that they happen while somebody is paying attention.
- Who should be reading transcripts that week?
- Whoever is allowed to change a rule. Reading without that authority produces a list of complaints; reading with it produces edits, and the difference shows by Wednesday.
- What if week one throws up something worse than these three?
- Then you found it in week one, with volume you can still handle by hand. That is the outcome the week is for, and it is the argument for launching on a live channel rather than a safe one.