Where the Next Sceptic Comes From
The instinct that catches a confidently wrong number was never taught. It was grown in the work we are automating first.
The 10,000-hour rule has been worn smooth by airport books. It was never quite that. Anders Ericsson, whose research the idea was built on, objected to the version that made it famous: “there is nothing special or magical about ten thousand hours,” he wrote. It was never just hours. It was deliberate practice — and, in the work I am describing, practice means repetition with feedback and a consequence when you get it wrong.
I still check a figure by hand that a spreadsheet gives me in three seconds. I trust the spreadsheet. I check it anyway — and the instinct that tells me when a number looks wrong before I can say why did not come from a course. It came from years of producing those numbers myself, and being wrong often enough to develop one.
And it is the first thing we are automating away — without noticing.
The thing that gets built while you are not looking
Every argument I make about using AI safely rests, in the end, on a person. Ground the numbers so a model reports facts instead of inventing them. Hand the arithmetic to a calculator so it cannot fumble the sums. I have written about both. But when the engineering is done, one risk is left that no amount of it removes: AI can still produce a clean, confidently wrong number. Catching that is not a technical job. It is a human one, and it needs the instinct.
Here is the uncomfortable part. That judgement is built, not taught. It is what Michael Polanyi called tacit knowing — we can know more than we can tell — and the better we get, the less we can explain it. The Dreyfus brothers’ five-stage model of skill acquisition — a description, argued over since, not a settled finding — has expertise moving away from conscious rule-following towards judgement the expert cannot easily put into words. The person who spots the wrong number often cannot tell you how they knew. They knew because they have seen ten thousand numbers, and this one sits wrong.
And it is grown in exactly the place AI clears out first — the routine, mechanical work. The invoices. The numbers that don’t reconcile. The reports nobody enjoyed. Dull, all of it. But the dullness was a training ground, and we are paving it over.
In a smaller business this lands harder, not softer. In the ones I work with, the instinct sits with one or two people — often the owner — and there is no deep bench it is quietly drawn from. So the shape of where AI is landing is worth sitting with. The clearest signal so far is not senior people being displaced — it is the hiring beneath them thinning. Stanford’s Digital Economy Lab, reading US payroll data to June, puts employment for twenty-two to twenty-five year olds in AI-exposed occupations 19% below where it would be had it kept pace with their less-exposed peers. Reduced hiring, not dismissals — and US data, not ours. The ones who catch AI’s mistakes are the experienced. The ones whose work AI can now do are the inexperienced. Which sets up a loop with no natural brake: better AI, fewer junior roles, fewer people who ever get the years in — and a decade on, fewer experienced people, because the experienced are only the inexperienced who stayed.
The comfortable answer, and why it doesn’t hold
The market’s answer is cheerful. The people who only move numbers from one box to another are simply done, and good riddance to the drudgery. I have some sympathy. Nobody built a career they were proud of by keying invoices, and no owner wants to pay a salary for work a machine does for pennies.
The reassuring next line is that the inexperienced will still learn — they will just learn by checking the machine’s work instead of doing it themselves. Review, not toil. It sounds right.
It isn’t. Reviewing output is not the same teacher as producing it. Lisanne Bainbridge named the problem in 1983 and called it an irony of automation: the system goes in because it does the job better than the person, and the person is then asked to monitor that it is working. Her subject was physical skill — it deteriorates when it is not used, she wrote, the refinements of timing going first. The extension to judgement is mine, not hers — but I would make it. A randomised trial published in February found something with the same shape. Fifty-two professional developers were set to learn an unfamiliar library, one group with AI assistance and one without. Tested afterwards without it, the AI group understood the code less well, debugged it worse, and had not, on average, been meaningfully faster for it. The learning was not banked, and the time was not saved either. Watching a correct-looking number appear teaches you nothing about why it is wrong. You learn that by being the one who got it wrong and carried the consequence.
Automate the routine — build the judgement on purpose
So the question is not whether to automate. It is what you do about the training ground you remove when you do.
The alternative is not keeping people busy with drudgery for its own sake. It is recognising that the drudgery was doing a second job — quietly manufacturing the judgement the business depends on — and that once it goes, that second job only gets done if you decide to do it.
Some firms that automated furthest are already adjusting. Ford hired 350 veteran engineers — some former employees, others from suppliers — after concluding its automated quality systems had underdelivered. Not a reversal of its own redundancies — experience brought in where the automation fell short. Commonwealth Bank of Australia replaced 45 service staff with a voice system, watched call volumes rise rather than fall, reversed the cuts, and admitted it had not been thorough enough about which roles were needed. IBM’s HR assistant resolves 94% of common employee questions without a person involved, on IBM’s own numbers; separately, IBM is tripling its entry-level hiring, and its chief human resources officer, Nickle LaMoreaux, put the reason in terms of the years ahead rather than this year’s cost line: “If we don’t continue to invest in entry-level hires, what happens in three to five years? There’s no pipeline.”
The survey numbers doing the rounds I have left out: the most quoted of them has no primary source behind it. What survives checking is older and narrower — in early 2025 Orgvue asked 1,163 senior leaders at large employers: 39% had made people redundant after deploying AI, and 55% of those said they got it wrong. Large firms, last year, not businesses your size. The named cases carry more than the percentages.
None of that is an argument against AI. It is an argument for adopting it with your eyes open. Automate the routine — that is where the cost genuinely comes out, and I would not talk anyone out of it. But keep people on the calls that carry risk, and put them deliberately in the way of the work that still builds judgement. Governed properly, AI frees your best people for exactly that. Left to run informally, it quietly retires the instinct you cannot buy back.
What I am not claiming is that any of this is new. Every wave of automation has been accused of hollowing out the bench, and the professions have generally re-grown their people on higher work. They may again. The difference is the speed, where the cut falls, and that the judgement used to arrive free — a by-product of work someone had to do anyway. Now it doesn’t.
The machine can do the work. It cannot yet do the knowing — and the knowing was always a by-product of the work. That leaves every owner a decision most have not realised they are making: how the business will keep producing the sceptic who can tell a wrong number from a right one. If that is a question you would rather answer deliberately than discover late, you know where to find me.