What is Hallucination

Large model hallucination refers to the phenomenon where the model generates content that appears plausible but is actually inconsistent with facts. It may manifest as: fabricating non-existent facts, distorting known information, or generating logically contradictory content. Hallucination severely impacts the credibility of AI applications.

Root Causes of Hallucination

  • Training data bias: Training data may contain erroneous, outdated, or incomplete information
  • Probabilistic generation nature: LLMs essentially predict the probability of the next token, rather than 'understanding' facts
  • Knowledge cutoff: The model cannot acquire new information after training is complete
  • Overgeneralization: The model overgeneralizes patterns from training data to mismatched scenarios
  • Contextual misleading: Incorrect information in user prompts may be adopted by the model

Mitigation Strategy 1: RAG Enhancement

RAG is one of the most effective methods to mitigate hallucination, constraining model output by retrieving external knowledge bases:

def rag_verify_response(question, llm_response, vectorstore):
    """Use RAG to verify the factual accuracy of the model's response"""
    # 1. Decompose the response into factual claims
    facts = extract_factual_claims(llm_response)

    verified_facts = []
    for fact in facts:
        # 2. Retrieve relevant evidence
        evidence = vectorstore.similarity_search(fact, k=3)

        # 3. Use LLM to verify consistency between fact and evidence
        verification = verify_fact_with_evidence(fact, evidence)

        if verification["supported"]:
            verified_facts.append((fact, "supported", verification["source"]))
        else:
            verified_facts.append((fact, "unsupported", None))

    return verified_facts

Mitigation Strategy 2: Self-Consistency

Improve reliability through multiple sampling and voting:

def self_consistency_check(prompt, llm, n_samples=5):
    """Generate multiple times and find the most consistent answer"""
    responses = []

    for _ in range(n_samples):
        response = llm.generate(
            prompt,
            temperature=0.7  # Use higher temperature to increase diversity
        )
        responses.append(response)

    # Find the most consistent answer
    # Can use semantic similarity clustering or voting
    consensus = find_consensus(responses)

    if consensus["confidence"] < 0.6:
        return {
            "answer": "I cannot determine the answer",
            "confidence": consensus["confidence"]
        }

    return consensus

Mitigation Strategy 3: Citation Tracing

Require the model to provide citation sources for each factual claim:

def request_citation_response(question, context_docs):
    prompt = f"""Based on the following reference materials, answer the question.

Reference materials:
{format_docs(context_docs)}

Question: {question}

Requirements:
1. Each factual claim must be annotated with citation source (e.g., [1], [2])
2. If the reference materials do not contain relevant information, explicitly state 'Not mentioned in the reference materials'
3. Do not fabricate facts not present in the reference materials

Please answer: """

    response = llm.generate(prompt)

    # Verify that citations actually support the corresponding facts
    verified = verify_citations(response, context_docs)

    return verified

Mitigation Strategy 4: Uncertainty Quantification

Let the model express its own uncertainty to help users judge credibility:

def uncertainty_aware_response(question):
    prompt = f"""Please answer the following question and assess the certainty of your answer.

Question: {question}

Please answer in the following format:
[Certainty] High/Medium/Low
[Answer] Your answer content
[Uncertainty explanation] If you are uncertain, please explain why

If certainty is 'Low', please suggest how the user can obtain more accurate information."""

    return llm.generate(prompt)

Metrics for Detecting Hallucination

  • Factual consistency: Whether the facts in the response are consistent with reliable sources
  • Logical consistency: Whether there are logical contradictions within the response
  • Citation accuracy: Whether the cited references actually support the corresponding facts
  • Refusal rate: Whether the model chooses to refuse to answer when uncertain
  • User feedback: Collect user evaluations of response accuracy

Summary

Hallucination cannot be completely eliminated, but it can be significantly reduced. The best practice is a combination strategy of RAG + citation tracing + uncertainty quantification. For high-risk scenarios (such as medical, financial), it is recommended to introduce human review.