Our numbers got worse in month two. Is something broken?

Check the calendar first. Month one is often a good month for unrelated reasons, and a business that compares month two against it will diagnose a fault that is actually a season.

How many rule edits per week is healthy?

Falling from week one, and never zero. Zero edits for a month means either that nothing in the business changed, which is unlikely, or that nobody is reading the handovers.

When can we stop paying attention?

The weekly hour never goes away, but by month three it stops producing surprises and turns into maintenance. If it never stops producing surprises, the rules are not the problem.

Contents5 sections

The baseline moved
Numbers that move for other reasons
What actually needs watching now
The enthusiasm problem
What it looks like when it goes right

Field notesARTICLE

The Second Month Is the Hard One

Month one has a clean comparison and an obvious to-do list. Month two has neither, and every number you were watching quietly changes meaning.

2026-11-13Costa3 min read

Short answer

In the second month the before-state stops being a valid comparison, because the baseline is now the assistant itself. Numbers start moving for reasons that have nothing to do with it — seasonality, a page change, a competitor — and the useful measurements shift from improvement to drift: how often the rules are edited, whether the same questions keep arriving, and whether anything in the rules has quietly stopped matching the business.

Key facts

Rule edits are recorded with dates, so the rate of editing is itself readable.
Recurring questions are counted, which is how a question that stopped being asked becomes visible.
Handovers are grouped by reason, so a change in one topic does not hide inside a stable total.
Every quote records the rule version behind it, which is what makes drift traceable after the fact.

The first month is easy to run because it has a clean comparison. Everything is measured against the way things were, the way things were is fresh in everybody's memory, and the to-do list writes itself out of the transcripts.

The second month has none of that, and it is where most of the value is either locked in or quietly lost.

The baseline moved

Every number worth watching had an obvious meaning in month one, because the comparison was to a business without an assistant.

In month two the comparison is to a business with one. Response time is not improving any more; it improved. Handover rate is not converging on anything; it is oscillating around whatever your rules imply. And a business that keeps comparing forward from month one will read normal variance as either triumph or failure, usually within the same fortnight.

Numbers that move for other reasons

This is the month the outside world reappears in the data.

A quiet fortnight is a season, not a fault. A spike in a single topic is a page that changed or a competitor's campaign. A rising handover rate in one category is new demand, and it is the most useful thing on the report because it arrived before anybody in the business noticed it by hand.

The discipline here is a boring one: before diagnosing anything, check whether the same week last year was also quiet, and whether anything on your own site changed. Most month-two panics resolve in ten minutes at that step.

What actually needs watching now

Three things, and none of them is response time.

The edit rate. How many rule changes went in this week. It should be falling from week one and it should never reach zero. A month with no edits means either that nothing changed in the business or that nobody is reading, and only one of those is plausible.

Questions that stopped arriving. A recurring question that disappeared is usually a page that got fixed, and it is the only evidence you will get that the fix worked. It is also the shape of the question asked forty times doing what it was supposed to do.

Drift. Somewhere in the rules is a number that changed in the business and did not change here. The way to find it is not a report; it is a monthly read of a handful of ordinary successful conversations, looking specifically for answers that are correct and out of date.

The enthusiasm problem

Month one has attention because it is new. Month two has the same workload and none of the novelty, and this is the point at which the weekly hour starts getting skipped.

It survives or it does not depending on whether it belongs to somebody by name. That is a separate question with its own answer, and it is the one that decides whether any of this is still true in March.

What it looks like when it goes right

By the end of month two the transcripts are boring, the edit list is short, and nobody in the company talks about the assistant much.

That last one is the real indicator and it is easy to misread as neglect. A tool that nobody discusses is either forgotten or working, and the difference between those two is exactly one hour a week of somebody reading handovers with the authority to change a rule.

Questions this raises

Our numbers got worse in month two. Is something broken?: Check the calendar first. Month one is often a good month for unrelated reasons, and a business that compares month two against it will diagnose a fault that is actually a season.
How many rule edits per week is healthy?: Falling from week one, and never zero. Zero edits for a month means either that nothing in the business changed, which is unlikely, or that nobody is reading the handovers.
When can we stop paying attention?: The weekly hour never goes away, but by month three it stops producing surprises and turns into maintenance. If it never stops producing surprises, the rules are not the problem.