Customer Contact 3.0

Back-Office Automation

Shrink 

Insight Reports

Few innovations in recent years have reshaped our world as rapidly as language models (LLMs). Their ability to converse in natural language and deliver seemingly coherent answers to any question thrown their way gives the 

 of true intelligence. But as we race to integrate them into workflows, it’s worth stopping to ask: are we giving them more credit than they truly deserve? 

This post explores one of the biggest limitations of LLMs: their tendency to hallucinate. It covers what hallucinations are, why they happen, and how to work around them. Because, as AI becomes more deeply embedded in how we live and work, understanding its weaknesses as well as its strengths will be key to realising its full potential.

Hallucinations occur when an LLM produces plausible 

 but incorrect information. Sometimes it’s a minor factual error resulting from inaccuracies in its training data. Other times it’s a complete fabrication – an entirely false statement which has no basis in its training data at all. Models have been caught inventing case law for legal briefs, generating fictional citations for academic papers, and convincingly summarising documents they’ve never seen. 

More troubling, perhaps, is the confidence with which these models present this incorrect information. Responses come across as smooth, authoritative, and impeccably structured –it’s easy to take them at face value. This is not by accident, but a function of how they are trained. 

LLMs are built to predict the most likely continuation of text, not necessarily the most accurate one

. Furthermore, they are fine-tuned to be agreeable, often conditioned to produce the response that feels most satisfying to the user. When faced with a question outside their knowledge, they fill in the gaps with content that sounds convincing.  

Of course, not all models hallucinate at the same rate and performance varies depending on the task. Benchmark tests, such as Vectara’s evaluation of models summarising news articles, show error rates ranging from under 1% to nearly 30%.  

Surprisingly, it isn’t simply a matter of picking the newest or most powerful model. In fact, so-called “reasoning models,” designed to break problems into steps, often perform worse. OpenAI itself has reported that its GPT-o3 and GPT-o4 models are substantially more prone to hallucinations than some earlier versions. 

Many experts now believe while efforts can be made to minimise hallucinations, they will never be fully eliminated. This is because errors are possible at every stage: training data is always incomplete, retrieval and classification are imperfect, and the generative process itself is inherently probabilistic. 

Although hallucinations may be here to stay, there are practical steps you can take to reduce their impact. 

RAG systems improve reliability by allowing an LLM to pull in fresh, relevant information before generating a response, rather than having it rely solely on its training data. They typically work by accessing a database or API to find documents related to your query, before passing that context to the model for use in its answer.

The field is moving fast, too. The French startup

 is developing an API that gives developers access to up-to-date web content from premium, trusted outlets, which can then be fed into an LLM to enrich its outputs. Knowledge graph technology such as 

’s GraphRAG takes this approach even further. Instead of feeding the model disconnected text snippets, a knowledge graph organises information into entities and relationships, mapping how facts connect to one another. This gives the model a much clearer picture of context to draw on and 

 to improve the accuracy of generalist LLMs like ChatGPT by over 25%.

Leverage vertical or domain-specific models

General-purpose LLMs are designed to handle a wide range of topics, which makes them more prone to hallucinations when venturing into niche areas. Domain-specific models, on the other hand, are fine-tuned on a set of data tailored to a particular industry or field. Because they’re trained on specialised knowledge, they can deliver outputs that are more accurate, consistent, and context-aware.

Early adopters include fields where reliability is most critical, such as law and medicine, but other industries are catching on too. BloombergGPT was custom trained for finance, while Walmart built Wallaby for supply chain and customer service tasks using decades of internal data. One startup in the field,

, trained its proprietary “Legal Pre-Trained Transformer” on over 150 million legal documents to achieve greater accuracy in analysing contracts and other legal content.

Build guardrails – both technological and human

As well as seeking to reduce the frequency of hallucinations, it is also important to build guardrails, both human and technological, that identify inaccuracies in outputs as early as possible.

, provide real-time guardrails to detect and mitigate hallucinations, while also addressing unsafe content, jailbreak attempts, and policy breaches. Tools like this act as automated checks, catching problems before they reach end users and giving teams greater visibility into model behaviour.

Not every safeguard needs to be advanced, though. Simple practices, such as asking the model to provide sources or explain its reasoning, are valuable when using LLMs for research. For important tasks, it’s also essential to independently verify that those sources actually exist — and that they say what the model claims. These habits not only catch errors, but also help you recognise the model’s blind spots, making it easier to apply the technology responsibly in the future.

Large Language Models may be inherently flawed, but that doesn’t make them useless. They will sometimes make things up, misrepresent the truth, or present information with misplaced confidence — but then again, so do humans. The key is not to expect perfection, but to understand how they work, recognise their limitations, and put strategies in place to mitigate their weaknesses.  

Our latest blog unpacks the common issue of Large Language Models and their tendency to hallucinate. Advisory Analyst Cara Hogg explores what they are, why they happen and offers practical strategies for working around them. 

Large Language Models: Smarter than us – or confidently wrong? 

Large Language Models: Smarter than us — or confidently wrong?

Stablecoins: Blockchain’s killer app or an economic chaos agent?

Long overshadowed by the noise of crypto and NFTs, stablecoins were, until recently, the lesser-known sibling in digital finance. Potentially functional but unsexy.  However, stablecoins have quietly started moving large sums of money — across borders, between corporates and through payment networks.  We explore what’s behind their acceleration, why they matter now, and what they could unlock for retailers and global brands. 

From Volatile to Viable: What Are Stablecoins?

Stablecoins are blockchain-based tokens designed to maintain a fixed value, typically pegged to fiat currencies like the US dollar or euro. Most are backed 1:1 by reserves — either cash, short-term Treasuries, or tokenised deposits. 

Where Stablecoins Start to Matter for Retail and Consumer

Consumer-facing businesses are exploring stablecoins for margin improvement, liquidity access and operational speed. These use cases stand out: 

Stablecoins aren’t frictionless — they come with real risks and structural challenges: 

The mood suggests though that stablecoins are no longer being treated like crypto experiments and more like infrastructure that needs regulating, not resisting. 

The Bigger Picture: Infrastructure, Not Just Innovation

Stablecoins won’t replace fiat. But they might quietly replace a good chunk of how fiat moves.  Whether embedded in loyalty, powering supplier payments, or enabling conditional refunds, the programmable layer is what unlocks value — not the asset itself. Ultimately, those experimenting have asked “Why are we still relying on slow, expensive, inflexible systems when alternatives exist?” 

Conclusion: The Quiet Layer Reshaping Payments

Stablecoins are not a silver bullet. But they are a credible alternative to traditional settlement rails — especially for businesses navigating cross-border complexity, liquidity needs, or cost pressure. What we are most excited about is the programmable aspect opening up opportunities for operational automation and loyalty drivers.  Retailers and brands would be wise to watch closely — or better yet, pilot quietly. 

Molly Walsh explores the acceleration of stablecoins, why they matter now and their potential for retailers and global brands. 

Advisory, reimagined

Trusted by the world’s best companies

We are outcome-focused

How we help

Technology Scouting

How we help

Strategy & Execution

How we help

Executive Support

Insight Reports

Technology Partnerships

Innovation Challenges

innovation stories

Insight

Large Language Models: Smarter than us — or confidently wrong?

Insight

Stablecoins: Blockchain’s killer app or an economic chaos agent?