When building enterprise-grade RAG systems, the performance bottleneck often lies not in the LLM's inference capability, but in the design of the retrieval pipeline. This tutorial focuses on balancing retrieval precision and end-to-end latency, covering key aspects such as the trade-off between sparse and dense retrieval, hybrid retrieval fusion, embedding model selection, index parameter tuning, re-ranking, and query rewriting. It provides runnable examples via the DeepSeek API to help advanced readers systematically optimize RAG performance.
Retrieval Precision Bottleneck: Trade-off Between Sparse and Dense Retrieval
Traditional BM25 relies on term frequency and inverse document frequency, excelling at exact keyword matching but failing on queries that are semantically related yet have no lexical overlap. Vector retrieval maps text into high-dimensional space via embedding models, capturing semantic similarity, but it is sensitive to proper nouns, IDs, abbreviations, and is heavily influenced by domain distribution. In real scenarios, user queries may mix both types of needs, and a single retrieval mode inevitably leads to insufficient recall.
For example, in the medical domain, the query "preventive effect of aspirin on myocardial infarction" and the document "application of ASA in secondary prevention of ACS" have almost no lexical overlap, resulting in a very low BM25 score, yet they are highly semantically related. Conversely, for the query "CT scan", BM25 can precisely hit documents containing "CT", while vector retrieval might miss them due to vector space offset. Therefore, hybrid search becomes a necessary choice to improve recall. We recommend prioritizing hybrid strategies in scenarios involving specialized terminology, diverse user queries, or mixed Chinese-English corpora.
| Dimension | BM25 | Vector Retrieval |
|---|---|---|
| Matching Mechanism | Term frequency-inverse document frequency | Semantic vectors (dense) |
| Recall Advantage | Exact lexical matching | Semantic relevance matching |
| Lexical Overlap Sensitivity | High | Low |
| Domain Applicability | General | Requires adaptation to domain data |
| Index Construction | Inverted index | Vector index (ANN) |
Hybrid Retrieval Strategy: RRF Fusion and Weight Tuning
The core of hybrid retrieval is fusing results from different retrievers. The most common method is Reciprocal Rank Fusion (RRF), which scores documents based on the reciprocal of their ranks across retrievers. The formula is: score(d) = Σ_{r∈R} 1/(k + rank_r(d)), where k is a smoothing constant (commonly 60) and R is the set of retrievers. RRF does not require score normalization and is robust to rank anomalies.
For weight tuning, weights w_r can adjust the contribution of each retriever. For example, when exact term matching is more important in domain data, increase the BM25 weight; conversely, when semantics are emphasized, increase the vector weight. We can optimize weights via grid search on a validation set, using metrics like Recall@k and MRR. In engineering, we recommend using Ray or Optuna for hyperparameter search to avoid manual tuning.
The following example demonstrates how to call the DeepSeek API to obtain embeddings and compute RRF scores (assuming retrieval result lists are already available):
import requests
import numpy as np
def get_embedding(text):
resp = requests.post(
"https://api.deepseek.com/v1/embeddings",
headers={"Authorization": "Bearer your-deepseek-api-key"},
json={"model": "text-embedding-ada-002", "input": text}
)
return resp.json()["data"][0]["embedding"]
def rrf_fusion(result_lists, k=60, weights=None):
scores = {}
for idx, doc_list in enumerate(result_lists):
w = weights[idx] if weights else 1.0
for rank, doc_id in enumerate(doc_list):
scores[doc_id] = scores.get(doc_id, 0) + w / (k + rank + 1)
return sorted(scores.items(), key=lambda x: -x[1])
# Example: Fusing BM25 and vector results
bm25_results = ["doc1", "doc3", "doc4"]
vector_results = ["doc2", "doc1", "doc3"]
fused = rrf_fusion([bm25_results, vector_results], weights=[0.5, 1.0])
print(fused)Embedding Model Selection: From BGE to LLM-Embedder
The embedding model determines the upper bound of vector retrieval. Mainstream models include the BGE series (BAAI/bge-large-zh), M3E, and OpenAI's text-embedding-ada-002. In Chinese scenarios, BGE performs well on the C-MTEB benchmark, but domain data still requires fine-tuning. LLM-Embedder is specifically designed for retrieval scenarios, trained via contrastive learning, and excels on long-tail queries.
Selection recommendations:
- If resources are sufficient, prioritize BGE-large-zh or LLM-Embedder, as they outperform smaller models in semantic matching.
- For multilingual needs, consider M3E or BGE-m3, which support cross-lingual retrieval.
- When fine-tuning, use domain-specific query-document pairs, adopt the InfoNCE loss, and pay attention to negative sample mining strategies (e.g., hard negatives) to improve discriminability.
Empirical comparison (on a custom legal dataset, nDCG@10): BGE-large-zh 0.61, M3E-base 0.58, text-embedding-ada-002 0.55. After fine-tuning, BGE improves to 0.68.
Index Construction Optimization: Tuning IVF and HNSW Parameters
Vector index directly determines retrieval latency and recall. IVF partitions the space via clustering, while HNSW uses a multi-layer graph structure for efficient approximate search. Key parameters:
- IVF nlist (number of cluster centers): If nlist is too small, each list becomes large, increasing scan cost; if too small, partitioning is coarse, reducing recall. Typically, nlist = 4*sqrt(N) ~ 8*sqrt(N), where N is the number of documents.
- HNSW M (maximum number of connections per layer): Larger M means denser graph connections, improving recall but increasing memory and build time. Common M=16~32.
- efConstruction (dynamic list size during construction): Controls build quality; larger values yield better index quality but slower construction. Common 100~200.
- efSearch (dynamic list size during query): Directly affects query precision and latency, adjustable online.
In engineering tuning, we typically follow: first set a recall target (e.g., 95% Recall@10), then gradually adjust parameters. The following example uses faiss to build an IVF-HNSW hybrid index and evaluate parameter impact:
import faiss
import numpy as np
# Generate random vectors to simulate embeddings
d = 128
n = 100000
xb = np.random.random((n, d)).astype('float32')
# Build IVF-HNSW index (actually using HNSW instead of IVF, here demonstrating IVF_HNSW)
quantizer = faiss.IndexHNSWFlat(d, 32) # M=32
index = faiss.IndexIVFFlat(quantizer, d, 100, faiss.METRIC_L2) # nlist=100
index.train(xb)
index.add(xb)
# Query parameters
index.nprobe = 10 # number of clusters to probe during query
xq = np.random.random((1, d)).astype('float32')
D, I = index.search(xq, 10)
print('Top-10 indices:', I[0])
print('Distances:', D[0])Re-ranking Models: Precision Improvement and Cost of Cross-Encoder
In the re-ranking stage, a Cross-Encoder model is used,For example, BGE-reranker-large concatenates the query and document as input and outputs a relevance score. Compared to the vector inner product of Bi-Encoder, Cross-Encoder captures finer-grained interactions, significantly improving accuracy, but with high inference overhead (each pair requires one forward pass).
In the RAG pipeline, we typically retrieve the Top-100 candidates first, then use a reranking model to select the Top-10 for the LLM, balancing precision and cost. If the budget is limited, consider using small model distillation or latency optimization (e.g., batch inference, caching). Measured data: On the MS MARCO dataset, Cross-Encoder achieves an MRR@10 that is 8-10% higher than Bi-Encoder, but inference time increases by 50 times.
The following example shows how to implement cross-encoder reranking using the DeepSeek API (using the deepseek-chat model for scoring, note the prompt mode):
import requests
def cross_encoder_score(query, doc, api_key="your-deepseek-api-key"):
prompt = f"Please judge the relevance between the query and the document, output a score from 0 to 1:\nQuery: {query}\nDocument: {doc}\nScore: "
resp = requests.post(
"https://api.deepseek.com/v1/chat/completions",
headers={"Authorization": f"Bearer {api_key}"},
json={
"model": "deepseek-chat",
"messages": [{"role": "user", "content": prompt}],
"temperature": 0
}
)
return float(resp.json()["choices"][0]["message"]["content"].strip())
query = "How to prevent Alzheimer's disease?"
doc = "Maintaining cognitive training and a Mediterranean diet can help reduce the risk of Alzheimer's disease."
score = cross_encoder_score(query, doc)
print(f"Relevance score: {score}")Query Rewriting and Expansion: Upstream Methods to Improve Recall
Query rewriting generates sub-questions or variants to cover different retrieval angles. For example, the user query "climate change impacts" can be rewritten as "impacts of climate change on agriculture" or "climate change and extreme weather". Query expansion supplements the retrieval formula with synonyms and related terms, such as expanding "GDP" to "gross domestic product".
Leveraging LLM capabilities, we can build a rewriting chain: first let the LLM generate multiple sub-queries, retrieve results for each, and then merge the results. However, note the increased latency from rewriting; it is recommended to trigger this only when retrieval results are sparse.
In engineering implementation, you can use the DeepSeek API for query rewriting. The following example generates sub-questions:
import requests
def rewrite_query(original, api_key="your-deepseek-api-key"):
prompt = f"Please rewrite the following query into 2 more specific sub-questions, one per line:\n{original}"
resp = requests.post(
"https://api.deepseek.com/v1/chat/completions",
headers={"Authorization": f"Bearer {api_key}"},
json={
"model": "deepseek-chat",
"messages": [{"role": "user", "content": prompt}],
"temperature": 0.3
}
)
content = resp.json()["choices"][0]["message"]["content"]
return [l.strip() for l in content.split('\n') if l.strip()]
queries = rewrite_query("How to optimize the training speed of deep learning models?")
for q in queries:
print(q)Context Compression: Reducing Noise and Token Consumption
Retrieved documents often contain a lot of irrelevant content. Directly concatenating them into the LLM dilutes attention and increases token consumption. Context compression aims to extract the most relevant segments to the query and condense the expression. Common methods include rule-based extractive compression (e.g., key sentences), summary-based generative compression, or using small models (e.g., DistilBERT) to judge sentence-level relevance.
In RAG, we typically extract the top-k relevant sentences from each document, merge them, and truncate by length. For example, using a simple heuristic: compute the cosine similarity between sentences and the query, sort, and take the top 3 sentences. If full semantic preservation is needed, you can use an LLM for summarization, but this introduces additional latency.
Evaluation results show that good compression can reduce LLM input length by 60% while retaining 95% of key information, with only a 1.5% drop in the F1 score of the final generated answer.
Continuing from the previous analysis of retrieval precision and reranking strategies, this section will delve into end-to-end performance optimization along the engineering mainline, from caching to architecture, and provide actionable selection and tuning recommendations. All suggestions are based on real business scenario stress tests and production practices, and code examples are based on the DeepSeek API.
Cache Mechanism Design: Semantic Cache and Exact Cache
Caching is the most direct way to reduce latency, but RAG systems need to distinguish between two types of caches: exact cache (exact match) and semantic cache (semantic cache). Exact cache uses the hash of the query text as the key and directly reuses results. It is simple to implement and can achieve a hit rate of 30%-50% (in idempotent scenarios), but it is ineffective for paraphrased queries. Semantic cache, on the other hand, maps queries to vectors via an encoder and retrieves similar items in the cache library. Upon a hit, it can reuse the answer or directly return similar cached content. The challenge of semantic cache is balancing the similarity threshold—if the threshold is too high, the hit rate is low; if too low, it may return irrelevant content. In practice, a dual-threshold approach is often used: a high threshold (e.g., cosine > 0.95) directly reuses results, while a medium threshold (0.85-0.95) only returns cached evidence snippets, and the LLM regenerates the answer.
From an engineering implementation perspective, exact cache can use Redis with TTL, and the cache key should include query + top_k + rerank_model to avoid dirty data from parameter changes. Semantic cache requires introducing a vector index (e.g., FAISS) and storing both the original query vector and the cached vector. In terms of latency benefits, exact cache hits can reduce end-to-end latency by about 60% (from 1.2s to 0.5s), while semantic cache hits reduce it by about 40% (due to vector retrieval). However, note cache penetration and avalanche—it is recommended to preheat popular queries and set random expiration times. Additionally, caching is only suitable for deterministic processes; if the answer depends on user context, it cannot be reused, so user dimensions should be added to the cache key.
Parallelization and Pipelining: Architecture to Reduce End-to-End Latency
The three stages of RAG (retrieval, reranking, generation) have natural dependencies, but pipeline parallelism can be designed: split the queries into shards, each shard runs the full process independently, and then merge the results. A more refined approach is stage asynchrony: while the retrieval stage is not yet complete, simultaneously start lightweight pre-generation (e.g., generating a summary prefix), and correct it after the evidence arrives. But the core benefit comes from eliminating idle waiting: in a traditional synchronous flow, each stage occupies 100% of the serial time, and the total latency is the sum of the three stages. In contrast, a pipeline design allows stages to overlap at the micro-batch granularity, making the overall latency close to the maximum stage duration. For example, dividing 100 queries into 4 batches of 25 each, with retrieval averaging 80ms, reranking 50ms, and generation 400ms, the serial total latency = 4×(80+50+400)=2120ms, while a 4-stage pipeline (ideal static scheduling) has a total latency ≈80+50+400+3×max(400,80,50)=1930ms, saving only 9%. However, if the generation stage can stream output, the savings can exceed 20%.
In implementation, Python can use concurrent.futures or asyncio, but cross-stage coordination requires attention to backpressure. A more robust solution is to use a message queue (e.g., Redis Streams) to connect stages and deploy consumer services independently. Below is a simple asynchronous pipeline example:
import asyncio
from openai import AsyncOpenAI
client = AsyncOpenAI(api_key="your-deepseek-api-key", base_url="https://api.deepseek.com")
async def retrieve(query: str):
# Simulate retrieval (actually call vector database)
await asyncio.sleep(0.2)
return ["doc1", "doc2"]
async def rerank(query: str, docs: list):
# Simulate reranking
await asyncio.sThe above practices are based on real project experience; adapt them to your own business needs. RAG optimization is an endless journey, and we hope this article serves as a performance accelerator for your system.