Few innovations in recent years have reshaped our world as rapidly as language models (LLMs). Their ability to converse in natural language and deliver seemingly coherent answers to any question thrown their way gives the 

 of true intelligence. But as we race to integrate them into workflows, it’s worth stopping to ask: are we giving them more credit than they truly deserve? 

This post explores one of the biggest limitations of LLMs: their tendency to hallucinate. It covers what hallucinations are, why they happen, and how to work around them. Because, as AI becomes more deeply embedded in how we live and work, understanding its weaknesses as well as its strengths will be key to realising its full potential.

Hallucinations occur when an LLM produces plausible 

 but incorrect information. Sometimes it’s a minor factual error resulting from inaccuracies in its training data. Other times it’s a complete fabrication – an entirely false statement which has no basis in its training data at all. Models have been caught inventing case law for legal briefs, generating fictional citations for academic papers, and convincingly summarising documents they’ve never seen. 

More troubling, perhaps, is the confidence with which these models present this incorrect information. Responses come across as smooth, authoritative, and impeccably structured –it’s easy to take them at face value. This is not by accident, but a function of how they are trained. 

LLMs are built to predict the most likely continuation of text, not necessarily the most accurate one

. Furthermore, they are fine-tuned to be agreeable, often conditioned to produce the response that feels most satisfying to the user. When faced with a question outside their knowledge, they fill in the gaps with content that sounds convincing.  

Of course, not all models hallucinate at the same rate and performance varies depending on the task. Benchmark tests, such as Vectara’s evaluation of models summarising news articles, show error rates ranging from under 1% to nearly 30%.  

Surprisingly, it isn’t simply a matter of picking the newest or most powerful model. In fact, so-called “reasoning models,” designed to break problems into steps, often perform worse. OpenAI itself has reported that its GPT-o3 and GPT-o4 models are substantially more prone to hallucinations than some earlier versions. 

Many experts now believe while efforts can be made to minimise hallucinations, they will never be fully eliminated. This is because errors are possible at every stage: training data is always incomplete, retrieval and classification are imperfect, and the generative process itself is inherently probabilistic. 

Although hallucinations may be here to stay, there are practical steps you can take to reduce their impact. 

RAG systems improve reliability by allowing an LLM to pull in fresh, relevant information before generating a response, rather than having it rely solely on its training data. They typically work by accessing a database or API to find documents related to your query, before passing that context to the model for use in its answer.

The field is moving fast, too. The French startup

 is developing an API that gives developers access to up-to-date web content from premium, trusted outlets, which can then be fed into an LLM to enrich its outputs. Knowledge graph technology such as 

’s GraphRAG takes this approach even further. Instead of feeding the model disconnected text snippets, a knowledge graph organises information into entities and relationships, mapping how facts connect to one another. This gives the model a much clearer picture of context to draw on and 

 to improve the accuracy of generalist LLMs like ChatGPT by over 25%.

Leverage vertical or domain-specific models

General-purpose LLMs are designed to handle a wide range of topics, which makes them more prone to hallucinations when venturing into niche areas. Domain-specific models, on the other hand, are fine-tuned on a set of data tailored to a particular industry or field. Because they’re trained on specialised knowledge, they can deliver outputs that are more accurate, consistent, and context-aware.

Early adopters include fields where reliability is most critical, such as law and medicine, but other industries are catching on too. BloombergGPT was custom trained for finance, while Walmart built Wallaby for supply chain and customer service tasks using decades of internal data. One startup in the field,

, trained its proprietary “Legal Pre-Trained Transformer” on over 150 million legal documents to achieve greater accuracy in analysing contracts and other legal content.

Build guardrails – both technological and human

As well as seeking to reduce the frequency of hallucinations, it is also important to build guardrails, both human and technological, that identify inaccuracies in outputs as early as possible.

, provide real-time guardrails to detect and mitigate hallucinations, while also addressing unsafe content, jailbreak attempts, and policy breaches. Tools like this act as automated checks, catching problems before they reach end users and giving teams greater visibility into model behaviour.

Not every safeguard needs to be advanced, though. Simple practices, such as asking the model to provide sources or explain its reasoning, are valuable when using LLMs for research. For important tasks, it’s also essential to independently verify that those sources actually exist — and that they say what the model claims. These habits not only catch errors, but also help you recognise the model’s blind spots, making it easier to apply the technology responsibly in the future.

Large Language Models may be inherently flawed, but that doesn’t make them useless. They will sometimes make things up, misrepresent the truth, or present information with misplaced confidence — but then again, so do humans. The key is not to expect perfection, but to understand how they work, recognise their limitations, and put strategies in place to mitigate their weaknesses.  

Our latest blog unpacks the common issue of Large Language Models and their tendency to hallucinate. Advisory Analyst Cara Hogg explores what they are, why they happen and offers practical strategies for working around them. 

Large Language Models: Smarter than us – or confidently wrong? 

Large Language Models: Smarter than us — or confidently wrong?

At True, we’re backing entrepreneurs building the next generation of transformative Consumer businesses. One of them is Sava, 

who just announced their $19m Series A round

 and we’re thrilled to finally share why we wanted to back this team! 

Sava has developed the world’s first real-time, under-the-skin microsensor capable of reading multiple molecules such as glucose, lactate and cortisol – unlocking a new era of proactive, personalised healthcare.

Here’s some framing as to why we’re excited.

 consumers are increasingly aware and proactive about their overall health & wellness, demanding more personalised, real-time data to address both immediate health issues and improve longevity. 

 ageing populations and growing rates of lifestyle-related illnesses (like heart disease and diabetes) are putting immense pressure on healthcare systems worldwide. Preventative care can address issues earlier, leading to better outcomes and improved cost control. 

wearables are great for tracking calories, heart rate and sleep continuously, but they can’t read biomarker data from within your body. Equally, blood tests are painful and expensive, so tracking the health of your molecules isn’t practical on a regular basis.

 enter Sava. They have developed micro-sensing technology capable of detecting biomarkers in real-time, just under the skin. Their first product is focused on redefining the diabetic care market with Continuous Glucose Monitoring (CGM) – Sava’s biosensor is a pain-free, cost-effective and a highly scalable alternative to existing products.

Why we’re excited to be part of the journey

Sava are delivering genuine innovation in a market aching for improvement, and diabetic care is just the start. Their modular, multi-analyte sensing platform is capable of detecting additional molecules, which Sava plan to launch in the future. This is a huge leap forward in proactive, preventive and personalised healthcare.

Having raised over $30m to date, Sava are well positioned to push ahead with their plans to build advanced manufacturing capabilities and accelerate clinical validation of their first product.

We’re excited to be part of the journey with Ren, Raf and team!

Why we're excited about latest investment in Sava, who are pioneering real-time molecular health monitoring.

Why we invested in Sava 

 and Red On Sports have announced a new partnership, with Ribble joining as the presenting partner of the 2025 British Gravel Championships, set to take place from 12-14 September in Dalby Forest, North Yorkshire.

This new partnership brings together one of Britain’s longest-established bike brands with “the country’s most important gravel racing event – reinforcing a shared mission to champion the sport from grassroots participation through to elite competition”.

The deal aims to help to grow gravel in the UK, creating more visibility, opportunity and inspiration for riders at every level.

Speaking about the new partnership, the brand highlights that this “builds on Ribble’s continued investment in gravel, including the recent launch of two new specialist ranges, the ALLGRIT and ULTRA-GRIT; their podium-proven bikes currently being raced on by the Ribble Outliers team.”

Riders and spectators alike will have the chance to get hands-on with both ranges, with bikes available to test on the demo track.

Returning for its fifth edition, the British Gravel Championships presented by Ribble will take over North Yorkshire’s legendary Dalby Forest for a three-day celebration of gravel with riders of all levels competing on fast, thrilling and spectator-friendly courses.

: Gravel Time Trial – 13.8km solo course, 150m of climbing

: Masters & Junior Gravel Races – 113.6km, 2024m of climbing

: Elite Men & Women’s Gravel Races – 113.6km, 2024m of climbing with men’s and women’s national titles up for grabs

Alongside the racing, the weekend delivers an unmissable festival of gravel with the Sundowner social ride, panel talks, food & drink, live music, demos, and off-bike experiences. The 2025 Championships promise a weekend of world-class racing and an inclusive celebration of the gravel scene for both participants and supporters.

Ribble has been named the presenting partner for the 2025 British Gravel Championships at Dalby Forest this September. Building on the successful launch of their ULTRA-GRIT and ALLGRIT bikes, Ribble will be supporting its team, the Ribble Outliers, as they compete on the new models.

Hello, we’re True. We’re reimagining business through advisory and investment.

A unique model for investing, innovating & growing

Meet the changemakers

A portfolio that excites us. Every day.

True stories

Insight

Large Language Models: Smarter than us — or confidently wrong?

News

Why we invested in Sava

News

Ribble announced as presenting partner of the 2025 British Gravel Championships