The Number You Write Down Before You Start
Ask an owner who has put AI into the business how it went, and the answer comes back positive. The demo landed. People are using it. It is saving time. Everyone nods and the meeting moves on to the next thing.
Then look at the management accounts. Nothing has moved. Or nothing you could point at and say that was this.
That is not dishonesty. It is what happens when nobody wrote anything down beforehand. Afterwards there is no number to be wrong against. So every impression counts towards the success, and the person with the least reason to look for a contradiction is the person who approved the spend.
The mechanism, which is duller than it sounds
Before the build, you write down two things. What the job costs you now. What the system would have to do to be worth keeping. After the build, you measure the first one again.
The difference between those two numbers is proof. A number found afterwards is a story.
That is the whole of it, and the reason it does not get done is that it looks too small to bother with. A baseline in a business your size is not a data project. It is a line in a notebook. Four hours a month of somebody’s Tuesday. Eleven days to close. Two in ten quotes come back for correction. Fifty pounds a run to an outside supplier. Pick whichever of those your business actually produces. It is already happening, and nobody has counted it, because nobody has needed the figure before.
The criterion is the same size. This takes four hours a month, and whatever we build has to produce output that matches the spreadsheet. One line. Written before the work starts, when you still have no idea whether it will work. Which is exactly what makes it worth anything.
Vin Vashishta, who writes about the economics of AI, put a version of this from the far end of the market in August, looking at enterprises rather than businesses of your size. What the analyses of failed projects keep finding, he says, is that most had no agreed definition of success before the work started, and nobody measuring the return after it launched.
He says, and I would put that on the wall: “Ship something and monetize it, even if the numbers are small. Profitability is your best credential.”
Small and real beats large and projected. That is not modesty. It is the only version anybody can check.
Why it keeps not happening
Because the projection arrives before the build, and it does the job the measurement was supposed to do. Somebody has already said what this will save. It has been in a paper, or a proposal, or a conversation with a supplier. And it has done its work. The money moved. Measuring afterwards feels like turning up to check a decision that was made months ago.
And afterwards, the person marking the homework is the person who bought the system. Nothing sinister in that. It is simply that the value question sits in a seat nobody has been given. Finance did not choose it. Operations did not scope it. The person who championed it has every ordinary human reason to see it working, and no reason at all to go looking for the number that says otherwise.
So the question in the room becomes “Is the AI working?”, which has no honest answer, and which your supplier will answer for you anyway.
Where I learned this, which was not from a client
Until July I had a favourite piece of arithmetic. The line was that an agent I had built ran for about two pounds a go, against an outside service at about a thousand pounds a month. It was a good line. It was written into my own house rules as the example of how to earn a point through specificity rather than adjectives, which tells you roughly how attached to it I was.
In July I stopped using it. Nobody challenged it. I went looking for the run behind it and there was not one. No production run, no measured cost, no before-figure recorded anywhere. What I had was a calculation I had done in my head once and then repeated until it sounded like a finding.
My own rules say hyperbole excluded, the defensible claim only. I had broken them with the number I liked most, and I had done it in the document where the rule is written down.
So I am not offering this as somebody who has been doing it properly. I am offering it as somebody who found out he had not been. What I can tell you is what the absence cost me: a line I liked, and no way to know whether it was true.
What changes
The question stops being is the AI working and becomes what moved against the baseline. The second one has an answer, and anybody who can read the ledger can check it, including the people in your business who were sceptical in the first place, who are the ones worth convincing.
Then something quietly useful accumulates. Every build that starts with a written criterion ends as evidence whichever way it goes, and the ones that miss their number teach you more than the ones that were never given a number to miss. Being wrong against a line you wrote is fine. Not being able to tell is the expensive state, because it is the state in which you keep spending. And the before-figure only exists before. Once the build starts the old number is gone and cannot be reconstructed, which is exactly how mine went missing.
I have argued before that the moment your system gets stuck and hands over to a person is the most useful minute of its week. This is the same instinct pointed at the money instead of the judgement. Write the standard down first, then see what happens against it.
Three things I am not claiming
Not every job has a clean baseline. The more judgement a job carries, the worse a number describes it, and forcing one on it measures the wrong thing confidently. Where that is true, say so and pick a different job to start with.
Starting is not a method. Deciding to do this properly is evidence that I have started, not evidence that it works, and I have already made the mistake of treating one as the other.
And the discipline does not make the build work. It costs nothing and it changes nothing about the system itself. All it does is make the answer knowable which turns out to be the part that was missing.
If you are going to spend money on AI this quarter, the cheapest useful thing you can do is this, and you can do it before you speak to anybody: write one line about what the job costs you now, and one line about what a system would have to do to be worth keeping.
Ten minutes. I had not done it either.
If you would rather not end up marking your own homework, you know where to find me.