TL;DR

We investigated reducing hallucinations in medical question-answering with Llama-3.1-8B-Instruct.

Using Goodfire's Sparse Auto-Encoder (SAE) we identified neural features associated with accurate and hallucinated responses. Our study found that features related to the model’s awareness of its own knowledge limitations were available and useful in detecting hallucinations.

This was further demonstrated by steering the model using those features and reducing hallucination rates by almost 6%. However, we observed that larger models exhibit greater uncertainty, complicating the distinction between information clearly known to the model and that which is clearly unknown.

These findings support research that features learnt during fine tuning, allow the model to learn about its own knowledge, such features are useful in identifying lack of knowledge.

However, despite these feature-based approaches having promise for AI safety, they require substantial refinement for real-world medical applications.

Introduction

What if your doctor relied on an AI model that confidently gave you the wrong medical advice? As AI systems increasingly assist in high stake fields like medicine, their potential to revolutionize the medical industry comes with significant risks. One urgent issue is hallucination, this is when an AI model confidently generates false or misleading information. In a medical context, these errors can have severe consequences, from misdiagnoses to recommending inappropriate treatments.

At the recent Mechanistic Interpretability Hackathon, we explored how to reduce hallucinations in Llama-3.1-8B-Instruct, focusing on medical question-answering. Using a production-grade tool, namely Goodfire’s Sparse Auto-Encoder (SAE) via API, we identified the features associated with both grounded and hallucinated responses from the LLM. With these features, we trained a classifier to predict hallucinations in real-time on the MedHalt FCT dataset, which we then used to steer the model, thereby reducing hallucinations.

In this blog we will share our findings, explain the methodology used, tie our work to a paper called “Do I know this entity” Fernando et al. and reflect on how feature activations can be used to enhance the safety and reliability of AI in medical applications.

Key Concepts: Sparse Autoencoders, Features, Feature Steering and Hallucination Robustness

S...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments