Generative AI Is Not an Expert: The Decision Intelligence Gap Your AI Strategy Is Missing

Barely a week passes without a headline announcing that generative AI has disrupted another professional field that was once considered the exclusive territory of expensive human expertise. Law firms are deploying AI to replace junior associates. Strategy consultancies are reorienting their delivery models around AI-assisted analysis. Regulatory advisory practices are promising clients faster, cheaper answers through AI-powered research tools. The financial press covers these announcements with the breathless enthusiasm of a Lyle Lanley pitch meeting. Brockway got a monorail. Ogdenville got a monorail. North Haverbrook got a monorail. Hundreds of jobs. Millions saved. The future, arriving ahead of schedule. North Haverbrook could tell you how that worked out.

I don’t doubt that some of this is real. What I doubt is whether the organisations making these announcements, and the journalists covering them, have asked the question that actually matters: what kind of expertise are they replacing, and what does that mean for the quality of the decisions those experts supported?

That question does not appear in the press releases. It rarely appears in the board papers that approved the restructuring. And its absence is going to cause problems that will take considerably longer to fix than the cost savings took to realise.

Organisations want generative AI to help them make smarter decisions, faster. That is a reasonable ambition. The problem is that decision intelligence is not produced by a single type of expertise. It is produced by a combination of types, each operating at a different level and contributing something different to the quality of the final judgment. The AI strategy conversation in law firms, regulatory functions, and strategy practices is proceeding as though that combination does not exist, and as though replacing one layer of it with a faster, cheaper alternative leaves everything else intact.

It does not.

What the Dreyfus Model Tells Us About Generative AI and Expert Judgment

The psychologists Hubert and Stuart Dreyfus developed a model of skill acquisition that is relevant to this discussion. They identified five stages through which human expertise develops: novice, advanced beginner, competent, proficient, and expert. Each stage contributes something different to the quality of a decision. The critical insight is not the stages themselves. It is the observation that expert performance is not just faster or more accurate rule-following. It is a qualitatively different mode of engaging with a situation, and it contributes something qualitatively different to the decisions made within it.

A novice follows rules. Their contribution to a decision is compliance with established procedure. A competent practitioner applies hierarchical decision procedures, prioritises elements, and feels responsible for outcomes. Their contribution is structured analytical reasoning: decomposing the problem, processing the relevant information, generating a goal-directed output. These are valuable. They are also, broadly, what generative AI does well.

An expert simply knows what to do, without consciously applying rules at all. They perceive situations holistically. They know which argument will land in front of this judge, on this day, in this case. They know which regulatory approach will get a constructive response from this inspector and which will harden positions. That knowledge is not in any rulebook or training dataset. It is built through years of being genuinely present in specific situations, with real stakes, within specific relationships that carry history and consequence.

The decision intelligence problem, then, is not about choosing between human expertise and AI. It is about understanding which layer of a decision each is equipped to handle. Generative AI contributes to the analytical layer: systematic processing, pattern recognition, structured output generation. That is genuinely useful. But it is the competent layer, not the expert layer. And decisions in genuinely complex professional contexts require both.

You cannot put expert judgment in a system prompt. You cannot fine-tune your way to it.

AI Strategy Reversals: Klarna, Starbucks, and a Pattern Nobody Is Admitting

Before we arrive at the professional services context, it is worth pausing on a pattern that has been forming in sectors where the case for AI looked far more straightforward.

In 2024, the Swedish fintech company Klarna became the most-cited corporate proof point for AI replacing human workers at scale. The company announced that its AI assistant had replaced approximately 700 customer service agents, handling millions of customer interactions across dozens of languages. The financial press covered it as evidence that AI displacement of white-collar work had genuinely arrived. Within a year, Klarna was rehiring. Customer satisfaction had deteriorated on complex interactions. CEO Sebastian Siemiatkowski publicly acknowledged that the company had overestimated AI’s capabilities and underappreciated the human dimensions of service delivery. The announcement that had been celebrated as a strategic breakthrough was quietly repositioned as a learning experience.

Starbucks provides a more recent example from a different domain. In September 2025, the company rolled out an AI-powered inventory counting system across more than eleven thousand North American stores as part of its “Back to Starbucks” operational turnaround. The technology was designed to replace manual shelf counts with automated scanning, reducing the time baristas spent on inventory tasks. Nine months later, Starbucks retired it. The system frequently confused similar products, required workers to verify every scan, and in doing so created more work than it removed. One barista’s feedback to the company summarised the experience plainly: “The thought behind it was great, but the execution was proving difficult.”

The point here is not that AI failed entirely in either case. The critical point is that both deployments involved tasks that were genuinely procedural: scripted customer interactions, counting stock on shelves. These are exactly the kinds of tasks where AI’s analytical pattern-matching should be most effective. And even there, the gap between what the technology delivered in controlled conditions and what it delivered at operational scale was sufficient to force a retreat.

If that gap exists at the procedural level, organisations considering AI deployment in expert professional fields need to ask themselves honestly what they think changes when the tasks become genuinely complex, contextually sensitive, and consequential.

Generative AI Hallucination in Legal Proceedings: What the Courts Have Already Told Us

In the professional services context, the answer to that question is already on the record across multiple jurisdictions, including our own.

In 2023, a New York lawyer named Steven Schwartz filed a brief in a personal injury case against Avianca Airlines citing six prior court decisions as legal authority. None of those decisions existed. They had been generated by ChatGPT and submitted to the court as genuine precedent. When opposing counsel could not locate the cases, Judge P. Kevin Castel ordered the lawyers to explain themselves. They were sanctioned and the firm was fined five thousand dollars. The reputational damage was considerably more expensive.

In May 2026, the Insolvency and Companies Court in London handed down a judgment in Cork & Anor v Smith that should be required reading for any managing partner currently running an AI adoption programme. Pinsent Masons, one of the UK’s largest law firms, referred itself to the Solicitors Regulation Authority after a junior associate used a generative AI tool to research a point of insolvency law in what was, by any measure, a routine procedural application. The AI generated text purporting to be Insolvency Rule 12.37(5), a statutory provision suggesting the court had an express power to grant release to outgoing liquidators. The provision does not exist. ICC Judge Mark Mullen, noting that the junior associate had “almost entirely outsourced the thinking process” to the AI tool, issued a public admonishment, and noted that Pinsents had referred themselves to the Solicitors Regulatory Authority. Crucially, the AI had itself warned the junior lawyer to verify the statutory wording against authoritative sources before relying on it. The verification did not happen.

Closer to home, the Irish courts have been dealing with the same pattern of generative AI hallucination. In Reddan v An Bord Pleanála, a litigant in person seeking judicial review relied on a legal term the judge found “not familiar in Irish jurisprudence,” concluding it had “all the hallmarks of ChatGPT, or some similar AI tool.” In Coulston & Others v Elliott & Elliott, the High Court concluded that defendants had likely used generative AI to prepare submissions they could not explain in court, with the judge warning that “if they used a generative AI program, they have been fooled” and calling on courts to warn the general public against relying on generative AI in legal matters.

Most significantly for Irish practitioners, the Court of Appeal provided its first formal guidance on generative AI in litigation in Guerin v O’Doherty [2026] IECA 48, decided in March 2026. The defendant, acting in person, had used AI to prepare written submissions for a defamation appeal without informing the other side. Those submissions contained multiple hallucinated authorities that did not exist or did not support the propositions advanced. The Court dismissed all eight grounds of appeal and set out guidance that applies to all parties: AI may be used responsibly in preparing submissions, provided all parties and the court are expressly informed, and provided the party using it takes full responsibility for verifying the accuracy of everything placed before the court.

Who Is Actually Checking AI Outputs?

It is worth stepping back from the legal context for a moment, because litigation has a structural property that most organisational decision-making does not.

In court proceedings, there is always someone on the other side with a professional interest in finding errors. Opposing counsel checks citations because undermining the other side’s authorities is part of their job. Judges verify statutory provisions because the integrity of the proceedings depends on it. Professional obligations require solicitors and barristers to take personal responsibility for what they put before the court. These mechanisms are imperfect. The cases above show they catch AI failures after damage has already been done. But they exist. There is a built-in scrutiny function.

In the boardroom, the strategy meeting, the risk committee, or the regulatory compliance review, that scrutiny function is frequently absent. A recommendation lands in front of a decision-maker. It has been prepared using analytical tools that include AI. The structure is logical. The citations look authoritative. The argument flows. The question that rarely gets asked is: who has verified that the premises are sound? Who has checked whether the regulatory interpretation is accurate? Who has tested whether the market analysis reflects the actual situation rather than a plausible statistical approximation of it?

In many organisations, the honest answer is: the same team that produced it. Or nobody with the relevant expertise. Or, increasingly, a senior leader who has been progressively distanced from the analytical layer by the very restructuring their AI strategy enabled.

This matters because decision intelligence is not simply about having analytical inputs. It is about being able to trust them. And trust in expert outputs is not produced by the confidence of the output itself. It is produced by knowing that someone with appropriate expertise, accountability, and situational awareness has stood behind the analysis. That is what professional expert judgment provides. It is the difference between a well-formatted report and a report you can actually act on.

When an organisation replaces expert practitioners with AI systems at scale, it does not merely lose the analytical contribution those practitioners made. It loses the human layer that made the analytical outputs trustworthy in the first place. The AI produces the analysis. Nobody who is qualified to evaluate it remains to do so. The decision gets made. And if the analysis was wrong, the error does not surface in front of a judge who demands an explanation. It surfaces in a failed market entry, a regulatory sanction, a risk that was not what the model said it was, or a strategic direction that looked coherent on the slide deck and proved incoherent in practice.

The legal cases are useful precisely because the failures they document are visible and documented. The equivalent failures in organisational decision-making are typically neither.

What Decision Intelligence Actually Requires

Bring this back to what decision intelligence actually demands of an organisation, and the argument becomes straightforward.

An experienced consultant advising a board on the implications of a new AI Act obligation is not acting as a well paid retrieval system for legislative text. They know the regulators, this sector, and the informal history of how guidance gets interpreted in practice in the respective jurisdiction. They know which compliance posture will satisfy a review by downstream stakeholders. They know whether the legal text means what it appears to say or whether the enforcement practice has moved in a different direction. Most of that knowledge is never written down. It accumulated through years of direct engagement with specific institutions and specific people, allowing the expert advisor to apply insight in novel circumstances.

A senior strategy advisor helping an organisation assess a potential acquisition is not primarily serving an analytical function. They bring judgment about what the numbers do not say: the cultural fit, the integration risk, the market dynamics that the model has smoothed into an assumption. They have seen this kind of deal work and fail before, and that experience shapes what they look for and what they weight. The acquisition process may be procedural, but the adaptive expertise of the advisor helps identify pitfalls and plan for what happens the day after the ink has dried and the cheque has cleared.

A chief risk officer presenting to a board is not delivering an algorithm’s output. They are taking responsibility for a judgment call about what the organisation’s exposure actually is, what it means in context, and what should be done about it. That accountability is part of what makes the judgment trustworthy. It cannot be delegated to a system that bears no consequences for being wrong.

In each of these cases, the expert is not just generating analysis. They are vouching for it. They are providing the interpretive layer that converts analytical output into something a decision-maker can rely on. Remove that layer, and the decision-maker is left with outputs that look authoritative and carry no accountability. That is not decision intelligence. It is decision risk dressed up as decision support.

The AI Expertise Pipeline Problem

There is a short-term cost to getting this wrong and a longer-term one. The short-term costs, like the Avianca sanctions, the SRA referral and reputational damage to Pinsent Masons, and the wasted costs in the recent Irish cases, are visible and recoverable. Klarna discovered the medium-term cost in customer satisfaction scores. Starbucks discovered it in operational efficiency that moved in the wrong direction.

The longer-term cost is less visible but more serious. Expert practitioners are not built overnight. They develop through exposure to varied situations, through the accumulation of dilemmas that require genuine judgment, and through the feedback of real consequences in real contexts. Junior associates, trainees, and graduate analysts are not simply a cheaper tier of labour. They are the pipeline through which organisational decision-making capability is reproduced and developed over time.

In May 2023, IBM CEO Arvind Krishna was one of the most prominent voices in corporate America arguing that AI would replace back-office roles at scale, telling Bloomberg he could “easily see 30 percent” of certain functions being replaced by AI and automation over a five-year period. The company paused hiring for roles deemed AI-replaceable and deployed its AskHR system, which automated the majority of routine HR queries. The financial press treated it as a model for the industry.

By early 2026, IBM was tripling its entry-level hiring. Krishna told CNN the company expected to hire more graduates over the next twelve months than it had in the previous few years. The rationale, articulated explicitly by IBM’s chief human resources officer, was that cutting junior headcount creates an eventual shortage of mid-level and senior leaders. Bringing in experienced hires from outside to fill those gaps later is more expensive, takes longer, and produces people who do not understand the organisation’s systems, culture, or institutional knowledge. The cost saving at the junior level had turned out to be a down payment on a considerably larger problem further up the pipeline.

IBM is not a small organisation making a speculative bet. It is one of the longest-established technology companies in the world, with deep internal data on workforce dynamics. When IBM publicly reverses a position it publicly championed, it is worth paying attention to the reason it gives for doing so.

When you displace that pipeline, you stop producing the next generation of people who will have genuine expert judgment in ten years. You retain the senior practitioners who already know what to do in genuinely difficult situations. Those practitioners retire, or leave, or burn out from carrying a cognitive load that was previously distributed across a team. At that point you have a very efficient analytical layer and no one left who can tell the difference between an analytically plausible output and a wrong one.

That is not a technology problem. It is a governance and strategy problem.

Getting the AI and Human Expertise Mix Right

None of this is an argument against using generative AI in professional services or in organisational decision-making. The analytical layer — the retrieval, the pattern recognition, the structured processing of large bodies of text, data, and precedent — is exactly what AI does well. Organisations that are not using it for that purpose are leaving real value on the table.

The argument is about what sits above that analytical layer, and what happens when you remove it. Genuine decision intelligence requires the full combination: analytical capability to process and structure the available information, and situated expert judgment to evaluate what that information means for this situation, with these people, at this moment, and to take accountability for the conclusion. Generative AI covers one part of that combination well. It does not cover the other part at all.

Generative AI processes data. That data has architecture, context, metadata, and quality characteristics that determine whether the outputs are trustworthy, regardless of how sophisticated the model is. An AI drawing on a poorly structured or inadequately maintained data estate produces confident-sounding outputs that may be wrong in ways that never surface clearly. Unlike a fabricated court citation, there is no opposing counsel to flag it. It requires the human reviewing the output to recognise the error.

Addressing this requires expertise in disciplines many organisations have historically undervalued because they are not ‘shiny’: records management, metadata governance, information architecture, data lineage. These disciplines predate the modern GenAI hype by decades. They have often been treated as administrative overhead rather than as professional expertise worth developing and retaining.

They are now foundational prerequisites for the analytical layer to function reliably. Developing or rediscovering that expertise is as much a strategic imperative as any investment in AI tooling. An organisation that cannot answer basic questions about its own data ( such as what it means, where it came from, and whether it is fit for the purpose being asked of it) cannot trust what its generative AI tells it.

Measure Twice, Cut Once (or not at all)

Stripping out the human experts who provide the situated judgment layer does not give you AI-powered decision intelligence. It gives you faster processing with a judgment-shaped hole in the middle. Neglecting the data foundations beneath the analytical layer compounds the problem further. In litigation, that hole gets found by opposing counsel. In organisations, it may not get found until the consequences of the decision have already arrived.

Before restructuring a professional or advisory function around generative AI, an organisation needs to understand what decisions that function is actually enabling, and which layers of expertise those decisions depend on. Not what the job description says. What happens when things get complicated, ambiguous, or genuinely novel, and someone has to make a call that matters, and be accountable for it. The cost saving that looks compelling in a board paper may be the decision intelligence your organisation cannot afford to lose.

Lyle Lanley was on the road out of North Haverbrook before the monorail failed. That is worth remembering the next time a vendor presentation promises that generative AI will transform your professional functions, eliminate expensive expertise, and deliver decision intelligence at a fraction of the current cost. The organisations that will navigate this well are not the ones that moved fastest. They are the ones that asked, before the song ended and the cheque was signed, what exactly they were buying — and who would be accountable when the monorail stopped working.

---

Key takeaways for AI strategy and decision intelligence

• Generative AI operates at the competent level of the Dreyfus expertise model — analytical, systematic, rule-guided. It does not reach the proficient or expert level, which requires holistic situational judgment built through lived experience.
• Decision intelligence requires three layers: data foundations beneath the analytical layer, generative AI in the analytical layer, and expert human judgment above it. Removing any layer damages the others.
• The Klarna, Starbucks, and IBM reversals show that AI strategy failures occur even in procedural tasks. The risks are greater in expert professional contexts.
• AI hallucination in legal proceedings is not a series of isolated incidents. It is the predictable consequence of deploying procedural analytical competence in contexts that require expert judgment. Deploying Generative AI tools is increasingly seen as moving work from research and creation to review and oversight of AI outputs (which requires staff with more expertise).
• The expert pipeline matters. Organisations that remove junior staff to fund AI deployment stop producing the expert judgment they will need in ten years. IBM reversed course for exactly this reason.
• Replacing expert practitioners with generative AI does not produce decision intelligence. It produces faster processing with a judgment-shaped hole in the middle.
• The training of Generative AI models on Generative AI created content without expert review and quality assurance of that content poses a whole range of risks we first touched on back in 2023.
• The question of valuing expertise was looked at last month in this blog post.