What is the 5W Hallucination Index and What Did It Find?

In the evolving world of artificial intelligence, particularly with large language models like ChatGPT and Google Gemini, "hallucinations" are a critical challenge that users and developers must grapple with. AI hallucinations refer to instances where generative AI confidently asserts information that is false or unsupported by data. For family offices and ultra-high-net-worth (UHNW) canadianfamilyoffices.com investors who rely increasingly on AI for research, reporting, and decision support, understanding and measuring these hallucinations is essential to managing risk.

Introducing the 5W Hallucination Index

The 5W Hallucination Index is a novel framework designed to systematize the assessment of AI-generated hallucinations through the lens of factual fidelity and context. Named after the classic journalistic questions—Who, What, When, Where, and Why—it evaluates the accuracy and faithfulness of AI outputs based on these five fundamental dimensions.

The index serves as a diagnostic tool to quantify the prevalence and severity of hallucinations in AI communications. It contextualizes how often AI tools like ChatGPT and Google Gemini emit material errors that could mislead users, especially in high-stakes environments such as investment management and private markets due diligence.

Why the 5Ws?

• Who: Does the AI correctly identify key individuals or entities?
• What: Are the AI’s stated facts or events accurate and verifiable?
• When: Are timelines and temporal facts correct?
• Where: Are locations and settings accurately represented?
• Why: Are causal explanations or reasoning sound and supported?

These questions underpin reliable communication, making the 5W Index especially relevant for evaluating the trustworthiness of AI output.

Key Findings From the 5W Hallucination Index Research

Recent studies applying the 5W Hallucination Index to popular generative AI platforms have surfaced eye-opening insights:

1. Approximately 20 per cent of AI outputs contain material errors. In other words, one in five responses from ChatGPT or Google Gemini may include at least one factual statement that is materially incorrect or misleading. For family offices, this error rate poses significant operational risk if outputs are taken at face value without verification.
2. Private markets data scarcity exacerbates hallucination risk. Family offices rely heavily on AI to parse and summarize opaque data from private equity, venture capital, and alternative assets. Because these datasets are often non-standardized, incomplete, or confidential, AI models struggle to confidently generate accurate insights. Hallucinations in this context may arise from the model attempting to infer or guess missing information.
3. Hallucinations tend to be highly persuasive. AI-generated outputs present confident, fluent prose that can be mistaken for authoritative knowledge. This persuasiveness amplifies the risk, as users may not question or verify the hallucinated content, inadvertently creating a false sense of understanding or due diligence.
4. Faithfulness versus factual hallucinations. The 5W Index highlights a useful distinction: some hallucinations are unfaithful, meaning the AI distorts input data or misquotes source material, while others are factual, fabricating entirely new but untrue information. Both types undermine trust but require different mitigation strategies such as improved prompt engineering or external fact-checking.

Why AI Hallucinations Matter in Family Office Use Cases

Family offices and multi-family office teams are embracing AI tools to enhance productivity and analytical rigor, particularly in three high-impact areas:

1. Preliminary Research

AI tools help family office researchers rapidly gather background information on new asset classes, investment managers, or market trends. However, the hallucination rate means outputs can—without rigorous oversight—include inaccurate details that misdirect subsequent analysis or raise compliance concerns.

2. Parsing Complex Manager Reports

Generating concise summaries from lengthy private equity or hedge fund reports is another promising AI application. Yet these reports often use specialised jargon and nuanced data disclosures, creating fertile ground for hallucinations. The 5W Index findings highlight the importance of manual validation to ensure that “Who” and “What” are precisely depicted.

3. Summaries and Decision Support

Family office CIOs and investment committees may use AI-generated executive summaries to inform decision-making. Here, even a single material error regarding investment rationale (“Why”) or timing (“When”) could lead to costly strategic missteps.

Mitigating Hallucination Risks: Practical Strategies

Given these challenges, how should family offices incorporate AI without falling victim to hallucinations?

• Always ask, “What would we show an auditor?” Before adopting new AI tools or relying on outputs, consider if you could confidently present these results to compliance or audit teams. If the answer is no, tighten validation protocols.
• Layer AI outputs with human expertise. Treat AI-generated insights as a first draft or hypothesis that needs expert vetting, rather than definitive advice.
• Focus on faithfulness. Use prompts and tools emphasizing faithful summarization of source data rather than creative re-interpretations.
• Maintain a running log of hallucinations. Similar to keeping a "things AI got wrong" note, track recurring errors or patterns in hallucinations to inform training and governance.
• Demand transparency from AI vendors. Beware of vague claims like “uses the latest model.” Insist on detailed documentation about hallucination mitigation strategies and empirical error rates.

Comparing ChatGPT and Google Gemini Through the 5W Lens

Dimension ChatGPT Google Gemini Implications for Family Offices Who Usually accurate but prone to confusion with lesser-known entities Strong entity recognition enhanced by multimodal understanding Double-check unfamiliar names and verify identity against trusted sources What Can fabricate plausible but false facts Better at factual correctness but still susceptible to material error Cross-verify core facts before using for due diligence When Occasional timeline inconsistencies Improved temporality processing, though errors remain Validate event dates especially in regulatory or contract contexts Where Ambiguous location references common Enhanced geo-awareness but can hallucinate missing details Confirm location-critical information like fund domiciles or office addresses Why Tends to speculate beyond available data Stronger causal reasoning but not immune to unsupported explanations Check investment rationale against official documentation

Conclusion: The 5W Hallucination Index as a Governance Tool

As family offices deepen their engagement with AI platforms like ChatGPT and Google Gemini, the 5W Hallucination Index provides a structured way to understand and manage the persistent risks of hallucinations. The finding that roughly 20 per cent of AI-generated outputs contain material errors underscores the imperative for vigilance, especially in the context of private markets where data scarcity compounds uncertainty.

By framing hallucinations in terms of “Who, What, When, Where, and Why,” family office teams can develop targeted review and validation processes that uphold data integrity and support sound investment decisions. The balance between leveraging AI efficiency and maintaining rigorous human oversight will define the next generation of technology-enabled wealth management.

In a sector where trust is paramount and the cost of error is high, the 5W Hallucination Index is a timely beacon guiding the responsible adoption of generative AI.