AI Legal Research Tools Hallucinate 17-33% of the Time, Despite Vendor Claims

Why It Matters

This research highlights a critical gap between the marketing of advanced AI tools and their actual performance in high-stakes professional environments. Designers and engineers developing AI solutions must prioritize rigorous, independent evaluation over unsubstantiated claims to ensure user trust and mitigate risks.

Key Finding

Despite vendor assurances, AI legal research platforms frequently produce incorrect information, with hallucination rates ranging from 17% to 33%.

Key Findings

• AI legal research tools from LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) hallucinate between 17% and 33% of the time.
• Claims of 'hallucination-free' or 'eliminating' hallucinations by vendors are overstated.
• Substantial differences exist between systems in terms of responsiveness and accuracy.

Research Evidence

Aim: To empirically evaluate the reliability and hallucination rates of proprietary AI-driven legal research tools.

Method: Empirical evaluation with a preregistered methodology.

Procedure: The study designed and executed the first preregistered empirical evaluation of AI-driven legal research tools, specifically assessing hallucination rates and accuracy in legal citation and summarization.

Context: Legal research and AI-assisted legal practice.

Design Principle

Prioritize empirical validation and transparency in AI system development and deployment.

How to Apply

When evaluating or developing AI tools for professional use, establish clear metrics for accuracy and hallucination, and design testing protocols that mimic real-world usage scenarios to uncover potential flaws.

Limitations

The study focused on specific AI legal research tools and may not generalize to all AI applications or legal domains. The 'closed nature' of proprietary systems limits full transparency into their underlying mechanisms.

Student Guide (IB Design Technology)

Simple Explanation: Even the best AI tools for lawyers sometimes make things up, about 1 in 5 to 1 in 3 times, so you can't trust them completely without checking.

Why This Matters: This research shows that AI, even in professional fields, isn't perfect and can make mistakes. This is important for any design project that uses or creates AI, as you need to ensure your AI is reliable and safe.

Critical Thinking: Given the significant hallucination rates, what ethical responsibilities do designers and developers of AI tools have to inform users about these limitations?

IA-Ready Paragraph: This research highlights the critical issue of AI 'hallucinations' in professional tools, demonstrating that even advanced AI legal research platforms exhibit significant error rates (17-33%). This underscores the necessity for designers to implement robust validation processes and for users to maintain critical oversight, as AI outputs cannot be blindly trusted in high-stakes applications.

Project Tips

• When using AI for research, always cross-reference its output with original sources.
• Consider how to design systems that flag potential inaccuracies for user review.

How to Use in IA

• Reference this study when discussing the limitations of AI tools or the importance of user verification in your design project.

Examiner Tips

• Demonstrate an understanding of the potential pitfalls of AI, such as 'hallucinations,' and how these might impact user trust and product reliability.

Independent Variable: Type of AI legal research tool (e.g., Lexis+ AI, Westlaw AI-Assisted Research).

Dependent Variable: Hallucination rate (percentage of inaccurate information), accuracy of legal responses, responsiveness.

Controlled Variables: Specific legal research queries, dataset used for evaluation, methodology for identifying hallucinations.

Strengths

• First preregistered empirical evaluation of proprietary RAG-based legal AI tools.
• Development of a comprehensive dataset and typology for identifying AI hallucinations.

Critical Questions

• How can AI systems be designed to self-correct or flag potentially erroneous information more effectively?
• What level of hallucination is acceptable in different professional domains, and how can this be determined?

Extended Essay Application

• Investigate the potential for AI 'hallucinations' in a different professional domain (e.g., medical diagnosis, financial analysis) and propose design interventions to mitigate these risks.

Source

Hallucination‐Free? Assessing the Reliability of Leading <scp>AI</scp> Legal Research Tools · Journal of Empirical Legal Studies · 2025 · 10.1111/jels.12413