# [2502.17125] LettuceDetect: A Hallucination Detection Framework for RAG Applications
Abstract page for arXiv paper 2502.17125: LettuceDetect: A Hallucination Detection Framework for RAG Applications

# Computer Science > Computation and Language

[Submitted on 24 Feb 2025]

# Title:LettuceDetect: A Hallucination Detection Framework for RAG Applications

Authors: Ádám Kovács, Gábor Recski
View a PDF of the paper titled LettuceDetect: A Hallucination Detection Framework for RAG Applications, by \'Ad\'am Kov\'acs and 1 other authors
View PDF HTML (experimental)

> Abstract:Retrieval Augmented Generation (RAG) systems remain vulnerable to hallucinated answers despite incorporating external knowledge sources. We present LettuceDetect a framework that addresses two critical limitations in existing hallucination detection methods: (1) the context window constraints of traditional encoder-based methods, and (2) the computational inefficiency of LLM based approaches. Building on ModernBERT's extended context capabilities (up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach outperforms all previous encoder-based models and most prompt-based models, while being approximately 30 times smaller than the best models. LettuceDetect is a token-classification model that processes context-question-answer triples, allowing for the identification of unsupported claims at the token level. Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for example-level detection, which is a 14.8% improvement over Luna, the previous state-of-the-art encoder-based architecture. Additionally, the system can process 30 to 60 examples per second on a single GPU, making it more practical for real-world RAG applications.

Comments: | 6 pages
Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: | arXiv:2502.17125 [cs.CL]
| (or arXiv:2502.17125v1 [cs.CL] for this version)
| https://doi.org/10.48550/arXiv.2502.17125 arXiv-issued DOI via DataCite

## Submission history

From: Ádám Kovács [view email]
[v1] Mon, 24 Feb 2025 13:11:47 UTC (1,188 KB)
Full-text links:

## Access Paper:

View a PDF of the paper titled LettuceDetect: A Hallucination Detection Framework for RAG Applications, by \'Ad\'am Kov\'acs and 1 other authors
- View PDF
- HTML (experimental)
- TeX Source

view license
Current browse context:
cs.CL
< prev | next >
new | recent | 2025-02
Change to browse by:
cs
cs.AI

### References & Citations

- NASA ADS
- Google Scholar
- Semantic Scholar

export BibTeX citation Loading...

### Bookmark

Bibliographic Tools

# Bibliographic and Citation Tools

Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Demos
Related Papers
About arXivLabs
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)