Token Oversight: The hidden cost of non-expert systems

In May 2026, Judge Mullen in the High Court of England and Wales handed down a judgment that should be required reading for anyone who commissions, delivers, or supervises professional advisory services in any jurisdiction. The case, Cork & Anor v Smith [2026] EWHC 1199 (Ch), arose from a routine, uncontested block transfer application in insolvency proceedings. The kind of paperwork that gets dealt with as, well, paperwork. The kind of process that usually requires only token oversight.

The Case

A junior solicitor at Pinsent Masons used an AI tool to research a legal point about whether the court had power to grant a liquidator’s release on a block transfer order. The AI produced a statutory provision in response. It was complete, confident, correctly formatted, written in convincing legislative language. However, that language did not exist anywhere in the Insolvency Rules 2016. The tool had simply invented it. The solicitor incorporated this hallucinated text into a letter to the court, presenting it as a verbatim quotation from IR 12.37(5). The supervising senior associate reviewed the letter. The responsible partner approved it. Neither checked the text.

(The GenAI tool, for its part, had actually warned the solicitor multiple times that it could not verify the provision from a primary source. The user was advised to check the wording against legislation.gov.uk before being filed. The letter was sent anyway.)

The Outcome

What followed was, to put it gently, an ‘educational experience’ for all concerned. The judge queried the provision. Pinsent Masons sent a second letter attempting to explain the first. This too was drafted substantially with GenAI assistance. This compounded the problem with a post-hoc rationalisation that the judge described as “not credible.” Pinsent’s has referred itself to the Solicitors Regulation Authority in the UK. The judge’s conclusion was unambiguous: “Legal professionals bear ultimate responsibility for their work and cannot outsource the process of legal research or of legal reasoning to an AI. It is a tool to be used with caution.”

This was not a small firm with no AI policy, no supervision structure, and no experienced practitioners to catch the error. This was Pinsent Masons, with its own AI use policy that explicitly warned about hallucinations and required human review. The supervision and oversight failed anyway. It failed because the humans trusted the output of a tool they did not fully understand. They trusted it, perhaps under time pressure, but without checking the primary sources.

The Lesson

That structural failure where AI output treated as reliable without verification by someone with sufficient expertise to know whether it is right is not unique to legal practice or to Pinsent Masons. It is a risk that arises wherever AI is used as a primary delivery mechanism for professional advisory work rather than as a tool in the hands of qualified experts. And it is directly relevant to how organisations should think about procuring data advisory services.

The Commodity Illusion

There is a tendency, when procuring professional advisory services, to treat them like a commodity. You need an outsourced DPO, a data governance review, a data strategy engagement, or a data management maturity assessment. So you put it out to tender, line up the numbers, and select whoever comes in cheapest. Job done. Budget controlled. Procurement team happy.

The problem is that data advisory services, regardless of its form or format,  is not a commodity. The landscape it operates in has become materially more complex in recent years with changes in both technology and regulation. Despite the shuffling of the deckchairs in the EU Digital Omnibus it shows no signs of simplifying. In that environment, a race to the bottom on price carries risks that are not always visible in the tender evaluation spreadsheet. But these are the kinds of risk which have a habit of becoming very visible when things go wrong.

The Quality Lesson

W. Edwards Deming put it plainly: “He that would run his company on visible figures alone will soon have no company and no figures.” The visible figure in a tender evaluation is the price. The invisible figures are the quality of the knowledge behind the advice, or the rigour of the verification process underpinning that knowledge. They include the investment in staff training, development, and supervision to maintain that knowledge. They measure the hard won experience of the person whose name is on the output. Managing only by the first of those, while ignoring the others, is a strategy with a known failure mode.

A Hypothetical Example

If a submission for outsourced DPO services, for example, comes in dramatically below the others, it is worth asking: how is that possible? There are really only a few explanations. One is genuine efficiency. The smart use of technology and process can deliver real quality at lower cost. That happens, and when it does it is worth understanding how. The second is buying the work, where the provider is tendering low to win and hoping to make it up elsewhere once they win the work. The third, and the one that should concern us most, is that the delivery model involves material reliance on AI-generated outputs with limited expert oversight. It relies on junior or contractor staff using AI as the primary research and drafting tool, with light supervision and minimal quality control.

The Practical Reality

The first scenario is fine. The second is a commercial risk for the provider (or the your finance team when the contract starts to go massively over budget). The third is definitely the buyer’s problem, because the work product sitting in their files, that you are relying on to underpin your strategy, inform action, or assure compliance, was produced by a system that can, and does, simply make stuff up. And a human being with sufficient expertise to catch the error may not have been in the loop. Because humans, particularly experienced ones, can be expensive. Or, if the provider has oversold their humans because AI is helping, the humans in the loop may simply be too busy to properly check the generated outputs. Quality control is sacrificed for scale.

The Regulatory and Strategic Landscape Is Not Getting Simpler

This matters more now than it did several years ago. In data protection, GDPR enforcement has matured. Fines and enforcement action do happen. Litigation for data protection breaches is a live risk. The intersection of AI regulation with data protection law has created genuinely contested legal questions that require informed judgment, not template answers. Cross-border data transfers remain a source of real legal uncertainty. “Is this personal data?” is still a regular question.

In data governance and data strategy, the picture is equally demanding. Organisations are increasingly expected to demonstrate accountability not just in how they protect data but in how they manage it, quality-assure it, and use it to make decisions. AI governance, including the governance of AI systems that are themselves making or informing consequential decisions, has created new obligations and risks that did not exist in law five years ago. Data quality failures increasingly now have regulatory as well as commercial consequences.

The appropriate response to increasing complexity is increased investment in knowledge and experience (‘adaptive expertise’), not decreased spend and token oversight. If the landscape is harder to navigate than it was, the value of a skilled navigator goes up, not down. An AI-generated data governance framework that cites a standard that does not exist, or a maturity assessment that hallucinates its benchmarks (aka ‘makes it all up’), causes real harm. Even if nobody sues or no regulator takes notice, internally the error damages trust and credibility of the data protection team or data governance team. The consequences can just as easily be damage to internal operations and the bottom line.

The Economics of the AI Intern

There is a further dimension worth understanding. The cost side of AI-assisted professional services delivery is more complicated than it appears.

Running AI tools at any scale is not free. The cost of API tokens, the unit by which AI processing is metered, is real. Depending on usage patterns, that cost can be substantial. Tom’s Hardware reported recently that an unnamed company had accidentally spent $500 million on Claude AI in a single month, having rolled out licences to employees without putting usage limits in place. Half a billion dollars. In a month. That is an extreme case, but it illustrates that the economics of AI consumption can surprise you in both directions.

The practical response for a firm delivering services at a price point that undercuts the market is to set token limits, which constrains usage, and caps what each task or client matter can spend on AI processing. This is sensible cost management, but a crude form of ‘token oversight’.

Token limits mean that your AI assistant can run out of capacity mid-task. The context window closes. The analysis stops. The research being conducted at 4:55pm on the day before your deadline may not complete. Your AI ‘intern’, upon whom the delivery model depends, has effectively clocked off for the afternoon. And unlike an actual intern, it won’t stay late for the promise of a reference. 

The cost-control mechanism creates a reliability problem at exactly the moments when reliability matters most. That tension is embedded in any delivery model that relies on AI (or any automation) as a primary resource rather than as a tool in the hands of experienced professionals.

What You Are Actually Buying

When you procure data advisory services you are buying knowledge, judgment, and accountability. Knowledge of the law and the standards, awareness of how regulatory practice is developing, experience of how organisations at different stages of maturity have approached similar challenges. Judgment to weigh competing considerations and give you a view, not just a list of options. Accountability from a professional who will stand behind their advice and who has an obligation to check their work.

AI can assist all three. It cannot substitute for any of them. A service model that treats AI in effect as the primary practitioner and a human as the nominal signatory delivers the form of these things without the substance. The Pinsent Masons case is instructive precisely because it illustrates that this failure mode is not limited to inexperienced or under-resourced providers. It can arise in any organisation where AI output is trusted with token oversight and without the expert verification that gives it meaning.

A Legal Obligation to Look Harder

For Irish public sector bodies procuring data advisory services, the question of an abnormally low tender is not merely a matter of commercial judgment. It is a matter of legal obligation.

The Court of Appeal’s decision in Killaree Lighting Services Ltd v Mayo County Council [2025] IECA, confirmed this year, provides clear guidance on what contracting authorities must do when a tender price looks too good to be true. The case arose from a public lighting maintenance contract where Mayo County Council excluded Killaree’s bid on the basis that it was abnormally low. The Court of Appeal upheld that exclusion and set out the applicable principles with some precision.

The Duty to be Curious

Contracting authorities have a positive duty to investigate any tender that appears abnormally low. This arises regardless of whether they intend to reject it. The obligation to investigate is not triggered by a wish to exclude; it arises from the suspicion itself. Critically, contracting authorities are entitled to examine the constituent cost elements of a tender, not just the overall price. A tender that appears acceptable in aggregate may, on closer examination, contain individual line items priced in a way that raises serious questions about deliverability. The ‘duty to be curious’ introduces a need for purchasers to have more than token oversight over the validity of tender pricing.

The onus lies firmly on the tenderer to explain why their pricing is not abnormally low. Reference to satisfactory performance on other contracts at similar price points is not sufficient. The tenderer must explain how this contract can be performed at these rates. Maybe the tenderer has cracked a process efficiency. Perhaps they’re trying to ‘buy the work’. Maybe you’re just paying for a human signature on a machine output.

The lesson for public bodies tendering data advisory services is clear. If a submission for outsourced DPO services, a data governance review, a data management maturity assessment, or a data strategy engagement is priced at a level that cannot plausibly deliver the scope of work required (the legal analysis, the organisational assessment, the strategic framing, the benchmarking expertise) then the contracting authority is not only permitted to probe that, it is obliged to. Accepting an abnormally low tender without investigation is not prudent procurement. It is a risk in its own right.

The financial consequences of getting the process wrong clear from the Killaree case. But the more fundamental risk for any contracting authority that procures professional advisory services on price alone is a different one. It is the risk of receiving plausible advice that is wrong, and acting on it.

The Investment Argument

Data advisory work, in all its forms, is not the kind of work where the consequences of a wrong answer show up at the same time as the first invoice. They show up when a supervisory authority asks a question you cannot answer. They manifest when a data subject complaint surfaces an exposure you were told did not exist. They arise when a data strategy built on flawed assumptions fails to deliver. They materialise when a governance framework turns out to have been built on foundations that nobody ever verified or is simply unsuited to the size, scale, or culture of your organisation.

Deming was right. The figures you can easily see, such as the tender price, the day rate, the number of deliverables, are rarely the ones that determine whether the organisation’s investment in data pays off. The invisible figures are often the ones that matter. The depth of the knowledge behind the advice. The rigour of the verification. The experience and training that knows what to look for and what to check. The insight and intuition that spots a problem or identifies a solution

That is worth paying for. Token oversight is not.