Limitations of Basic RAG

Standard RAG systems perform poorly in the following scenarios: complex questions requiring cross-document reasoning, queries requiring structured filtering, and scenarios needing dynamic retrieval strategy adjustment. Advanced RAG techniques are designed to address these issues.

Multi-hop Retrieval

Multi-hop retrieval decomposes complex questions into multiple sub-questions, retrieving and reasoning sequentially to gradually approach the final answer:

from langchain.chains import MultiRetrievalQAChain

# Define multiple retrievers
retriever_docs = vectorstore_docs.as_retriever()
retriever_code = vectorstore_code.as_retriever()

# Multi-hop retrieval chain
chain = MultiRetrievalQAChain.from_retrievers(
    llm=llm,
    retrievers=[retriever_docs, retriever_code],
    retriever_descriptions=[
        "Document knowledge base: contains product docs and user manuals",
        "Code repository: contains source code and API docs"
    ]
)

The key to multi-hop retrieval is:

  • Query Decomposition: Break down user questions into independently retrievable sub-questions.
  • Intermediate Result Utilization: Use previous retrieval results as context for the next step.
  • Information Fusion: Integrate information from multiple sources into a coherent answer.

Self-Querying Retrieval

Self-querying retrieval lets the LLM automatically extract structured query conditions from user questions, combining semantic search with metadata filtering:

from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain.chains.query_constructor.base import AttributeInfo

metadata_field_info = [
    AttributeInfo(
        name="source",
        description="Document source",
        type="string"
    ),
    AttributeInfo(
        name="year",
        description="Document year",
        type="integer"
    ),
    AttributeInfo(
        name="category",
        description="Document category",
        type="string"
    )
]

retriever = SelfQueryRetriever.from_llm(
    llm=llm,
    vectorstore=vectorstore,
    document_contents="Technical document collection",
    metadata_field_info=metadata_field_info
)

# User asks "deep learning papers on RAG from 2024"
# Automatically constructs query: semantic="RAG deep learning" + filter year=2024, category="paper"
results = retriever.get_relevant_documents(
    "deep learning papers on RAG from 2024"
)

Contextual Compression

When retrieved documents are too long, contextual compression extracts the most relevant snippets:

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=base_retriever
)

# Returns only key content relevant to the query
compressed_docs = compression_retriever.get_relevant_documents(query)

Advanced RAG Architecture Design

Production-grade RAG systems typically adopt a layered architecture:

  • Routing Layer: Routes to different retrieval strategies based on query type.
  • Retrieval Layer: Combines multiple retrieval methods (keyword, semantic, structured).
  • Re-ranking Layer: Uses Cross-encoder to re-rank retrieval results.
  • Generation Layer: Generates final answers with retrieval results, supporting citation tracing.
  • Evaluation Layer: Continuously monitors the quality and performance of the RAG system.

Conclusion

Advanced RAG techniques move knowledge base Q&A from "usable" to "useful." It is recommended to start with basic RAG and gradually introduce advanced techniques based on actual pain points, rather than pursuing a "perfect architecture" from the start.