Overview

As Large Language Models (LLMs) become increasingly integrated into critical systems, detecting hallucinations reliably has emerged as a crucial challenge for AI safety. While much attention has focused on detecting hallucinations at either the token or sentence level, our research suggests both approaches miss important nuances in how LLMs generate and manipulate information. This post examines a specific challenge we've encountered in entity-level hallucination detection: the persistent problem of false positives across multiple detection methods.

The Current Landscape and Our Approach

Hallucination detection has traditionally operated at either the token level (examining individual words) or the sentence level (evaluating entire statements). Our research suggests an intermediate approach: focusing on entities - coherent semantic units like "artificial intelligence" or "Geoffrey Hinton" that carry discrete, verifiable pieces of information. This granularity aligns better with how information is structured and verified in practice.

To test this approach, we evaluated five SOTA uncertainty-based detection methods: Shifting Attention to Relevance (SAR), which weights token likelihood by semantic importance; Claim-Conditioned Probability (CCP), which recalculates likelihood based on semantic equivalence; standard Entropy and Likelihood measurements; and Stronger Focus, which combines likelihood, entropy, and attention-based propagation. We tested these methods against a custom dataset with careful entity-level annotations.

The False Positive Problem

Our most striking finding was consistent across all methods: they produce an unexpectedly high rate of false positives, often flagging legitimate content as hallucinated. This isn't simply a matter of oversensitive thresholds - the pattern persists across different calibration attempts and appears to be fundamental to how these methods process information.

Several clear patterns emerged from our analysis. Common entities, despite being correct, often trigger suspicion flags. Technical terms face heightened scrutiny even when used appropriately. Perhaps most tellingly, complex multi-token entities seem particularly prone to false positives. These patterns suggest deeper issues with how current methods aggregate and evaluate uncertainty.

Understanding the Challenge

We've identified three primary hypotheses about the ...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments