DeepSeek RAG Knowledge Base Setup Tutorial

Build an enterprise-grade knowledge base Q&A system from scratch using DeepSeek and LangChain. Document loading, vector embeddings, semantic retrieval, intelligent Q&A with source citations, complete code, ready to use.

Start Learning

DeepSeek RAG Knowledge Base

What is RAG and why do we need it?

RAG (Retrieval-Augmented Generation) is the core technology that enables large models to answer private knowledge. Without RAG, large models can only answer content from training data; with RAG, large models can answer accurately based on your documents, databases, and knowledge bases.

Step 1

RAG Principle Overview

Understanding how RAG works is a prerequisite for building a high-quality knowledge base. RAG consists of three core stages: Retrieval, Augmented, and Generation.

The Three Core Stages of RAG

Retrieval	Convert the user's question into a vector and find the most relevant document chunks in the vector database. This is the "search" stage of RAG.
Augmented	Concatenate the retrieved document chunks as context into the prompt. This is the "knowledge injection" stage of RAG.
Generation	The DeepSeek model generates an answer based on the augmented prompt. This is the "output" stage of RAG.

RAG Architecture Flow

The complete RAG architecture is divided into two phases: offline and online.

Offline Phase (Knowledge Base Construction):

Document Loading: Read various documents such as PDF, TXT, Markdown, web pages, etc.
Text Splitting: Split long documents into appropriately sized text chunks.
Vector Embedding: Convert text chunks into vector representations.
Vector Storage: Store vectors in a vector database such as ChromaDB.

Online Phase (Q&A):

User Query: Receive the user's input question.
Vector Retrieval: Convert the question into a vector and find the most similar text chunks in the database.
Context Concatenation: Use the retrieved results as context to construct an augmented prompt.
Model Generation: DeepSeek generates an answer based on the augmented prompt and includes source citations.

RAG vs Fine-tuning

Comparison Dimension	RAG	Fine-tuning
Knowledge Update	Real-time updates; adding or removing documents takes effect immediately.	Requires retraining; long cycle.
Cost	Low; only needs a vector database.	High; requires GPU training resources.
Interpretability	Traceable to specific documents.	Black box; difficult to trace knowledge sources.
Applicable Scenarios	Knowledge base Q&A, document retrieval, customer service systems.	Style imitation, domain-specific terminology, instruction following

For most enterprise scenarios, we recommend a combination of RAG and fine-tuning: use RAG for dynamic knowledge and fine-tuning to optimize model behavior. For more model details, see DeepSeek Model Architecture Explained.

Step 2

Environment Preparation

Install the necessary Python dependencies. The following command installs all libraries required for RAG development at once.

Install Core Dependencies

# Create virtual environment (recommended)
python -m venv rag-env
# Activate on Windows
rag-env\Scripts\activate
# Activate on macOS/Linux
# source rag-env/bin/activate

# Install core dependencies
pip install langchain langchain-community chromadb sentence-transformers openai

# Install document loaders (choose as needed)
pip install pypdf                  # PDF loading
pip install unstructured            # General document loading (PDF/Word/PPT/HTML)
pip install beautifulsoup4          # Web page parsing
pip install lxml                    # XML/HTML parsing acceleration

# Verify installation
python -c "import langchain; import chromadb; print('RAG environment ready')"

DeepSeek API Configuration

The RAG system uses the DeepSeek API as the generation model. Set an environment variable to store the API key:

# Windows PowerShell
$env:DEEPSEEK_API_KEY = "sk-your-api-key-here"

# macOS/Linux
# export DEEPSEEK_API_KEY="sk-your-api-key-here"

# Or configure directly in Python code
import os
os.environ["DEEPSEEK_API_KEY"] = "sk-your-api-key-here"

Note

If you use a locally deployed DeepSeek model via Ollama, no API key is needed; just point the API Base URL to http://localhost:11434/v1. See DeepSeek Local Deployment Tutorial.

Step 3

Document Loading

LangChain provides a rich set of document loaders that support various formats such as PDF, TXT, Markdown, web pages, CSV, and more. Choose the loader that fits your document format.

Loading PDF Documents

from langchain_community.document_loaders import PyPDFLoader

# Load a single PDF
loader = PyPDFLoader("docs/企业年报2025.pdf")
pages = loader.load()

# View loading results
print(f"Loaded {len(pages)} pages")
print(f"First page first 200 chars: {pages[0].page_content[:200]}")

# Batch load PDF folder
from langchain_community.document_loaders import DirectoryLoader

loader = DirectoryLoader(
    "docs/",
    glob="**/*.pdf",
    loader_cls=PyPDFLoader,
)
documents = loader.load()
print(f"Loaded {len(documents)} documents")

Loading TXT and Markdown

from langchain_community.document_loaders import TextLoader, UnstructuredMarkdownLoader

# Load plain text file
txt_loader = TextLoader("docs/产品手册.txt", encoding="utf-8")
txt_docs = txt_loader.load()

# Load Markdown file
md_loader = UnstructuredMarkdownLoader("docs/技术文档.md")
md_docs = md_loader.load()

Loading Web Pages

from langchain_community.document_loaders import WebBaseLoader

# Load a single web page
web_loader = WebBaseLoader("https://platform.deepseek.com/api-docs")
web_docs = web_loader.load()

# Batch load multiple web pages
urls = [
    "https://example.com/doc1",
    "https://example.com/doc2",
]
web_loader = WebBaseLoader(urls)
web_docs = web_loader.load()

Using Unstructured to Load General Documents

Unstructured is a powerful document parsing library that supports 20+ formats including PDF, Word, PPT, Excel, HTML, images, and more. Recommended for complex document scenarios:

from langchain_community.document_loaders import UnstructuredFileLoader

# Automatically detect file type and parse
loader = UnstructuredFileLoader("docs/产品方案.docx")
docs = loader.load()

# Batch load multiple formats
loader = DirectoryLoader(
    "docs/",
    glob="**/*.*",
    loader_cls=UnstructuredFileLoader,
)
all_docs = loader.load()
print(f"Loaded {len(all_docs)} document fragments")

Step 4

Text Splitting

Text splitting is a critical step for RAG quality. Too large splits lead to imprecise retrieval, too small splits lose context. LangChain's RecursiveCharacterTextSplitter is the most recommended text splitter.

RecursiveCharacterTextSplitter Core Parameters

Parameter	Description	Recommended Value
chunk_size	Maximum number of characters per text chunk	500-1000
chunk_overlap	Number of overlapping characters between adjacent chunks	50-200
separators	Splitting priority: first by paragraph, then by sentence, finally by character	Default is fine

Text Splitting Code

from langchain.text_splitter import RecursiveCharacterTextSplitter

# Create text splitter
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,        # Maximum 500 characters per chunk
    chunk_overlap=100,      # Overlap 100 characters between adjacent chunks
    length_function=len,
    separators=["\n\n", "\n", "。", "！", "？", ".", " ", ""],
)

# Split documents
chunks = text_splitter.split_documents(documents)
print(f"Original document count: {len(documents)}")
print(f"Chunk count after splitting: {len(chunks)}")

# View splitting results
for i, chunk in enumerate(chunks[:3]):

Best Practices for Text Splitting

Adjust chunk_size by content type: Use 500-800 for technical documents, 800-1200 for long articles, 300-500 for FAQ-like content
Do not omit chunk_overlap: Overlap prevents key information from being split at boundaries; recommended to set it to 10%-20% of chunk_size
Preserve metadata: Keep metadata such as document source and page number during splitting for later traceability
Chinese splitting: Add Chinese punctuation (。！？) to separators to ensure splitting at semantic boundaries
Load first, then split: Load the full document first, then split uniformly to avoid repeated loading and splitting

print(f"--- Chunk {i+1} ({len(chunk.page_content)} characters) ---") print(chunk.page_content[:200]) print()

Best Practices for Text Splitting

Adjust chunk_size by content type: Use 500-800 for technical documents, 800-1200 for long articles, 300-500 for FAQ-like content
Do not omit chunk_overlap: Overlap prevents key information from being split at boundaries; recommended to set it to 10%-20% of chunk_size
Preserve metadata: Keep metadata such as document source and page number during splitting for later traceability
Chinese splitting: Add Chinese punctuation (。！？) to separators to ensure splitting at semantic boundaries
Load first, then split: Load the full document first, then split uniformly to avoid repeated loading and splitting

Step 5

Vector Embeddings

Vector embeddings are the process of converting text into high-dimensional vectors (arrays of numbers). Texts with similar semantics have vectors that are closer in space. This is the core mathematical foundation for RAG retrieval.

Embedding Model Selection

Model Name	Dimensions	Chinese Support	Model Size	Recommended Use Case
BAAI/bge-large-zh-v1.5	1024	Excellent	1.3GB	First choice for Chinese documents
BAAI/bge-small-zh-v1.5	512	Excellent	96MB	Lightweight Chinese
sentence-transformers/all-MiniLM-L6-v2	384	Fair	80MB	English-focused
shibing624/text2vec-base-chinese	768	Excellent	400MB	Chinese semantic matching

Vector Embedding Code

from langchain_community.embeddings import HuggingFaceEmbeddings

# Use BGE Chinese embedding model (recommended)
embedding_model = HuggingFaceEmbeddings(
    model_name="BAAI/bge-small-zh-v1.5",
    model_kwargs={"device": "cpu"},       # Change to "cuda" if GPU available
    encode_kwargs={"normalize_embeddings": True},  # Normalize to improve retrieval accuracy
)

# Test embedding effect
test_text = "DeepSeek is a powerful open-source large language model"
embedding = embedding_model.embed_query(test_text)

Embedding Model Selection Recommendations

For Chinese documents, the BGE series models are strongly recommended, as they significantly outperform general English models in Chinese semantic understanding. For local deployment, use HuggingFaceEmbeddings to load them without API costs. If you pursue ultimate performance and stability, you can use cloud-based embedding APIs.

print(f"Vector dimensions: {len(embedding)}") print(f"First 5 values of vector: {embedding[:5]}") # You can also use DeepSeek API for embeddings (if officially supported) # Or use OpenAI-compatible embedding API from langchain_openai import OpenAIEmbeddings api_embedding = OpenAIEmbeddings( model="text-embedding-3-small", # Or use other compatible APIs base_url="https://api.deepseek.com/v1", api_key="sk-your-api-key-here", )

Step 6

Vector Database — ChromaDB

ChromaDB is a lightweight open-source vector database designed for LLM applications. No separate server is required; data is stored locally in files, making it perfect for small to medium-scale RAG applications.

Create Vector Database and Store

from langchain_community.vectorstores import Chroma

# Store the split documents into ChromaDB
vectorstore = Chroma.from_documents(
    documents=chunks,                    # Split text chunks
    embedding=embedding_model,           # Embedding model
    persist_directory="./chroma_db",    # Persistence storage path
    collection_name="deepseek_knowledge",  # Collection name
)

# Data is automatically persisted; load directly next time
print(f"Vector database now has {vectorstore._collection.count()} records")

Load Existing Vector Database

# Load existing vector database from disk
vectorstore = Chroma(
    persist_directory="./chroma_db",
    embedding_function=embedding_model,
    collection_name="deepseek_knowledge",
)

print(f"Loaded existing database with {vectorstore._collection.count()} records")

Similarity Search

# Basic similarity search
query = "How to call the DeepSeek model API?"
results = vectorstore.similarity_search(query, k=4)

for i, doc in enumerate(results):
    print(f"--- Result {i+1} (Source: {doc.metadata.get('source', 'Unknown')}) ---")
    print(doc.page_content[:300])
    print()

# Search with similarity scores
results_with_scores = vectorstore.similarity_search_with_score(query, k=4)
for doc, score in results_with_scores:
print(f"Similarity score: {score:.4f} | Content: {doc.page_content[:100]}...")

# MMR search (Maximum Marginal Relevance) — balances relevance and diversity
mmr_results = vectorstore.max_marginal_relevance_search(
    query, k=4, fetch_k=10, lambda_mult=0.7
)

Other Vector Database Options

FAISS: Meta's open-source vector indexing library, pure in-memory operations, extremely fast retrieval, suitable for millions of data points
Milvus: Enterprise-grade distributed vector database, supports billion-scale vectors, suitable for large-scale production environments
Weaviate: Open-source vector database with built-in vectorization and hybrid search, supports GraphQL API
Pinecone: Commercial vector database service, no maintenance required, suitable for teams that don't want to manage infrastructure
Qdrant: High-performance vector database written in Rust, supports filtering and group retrieval

Step 7

Build Retrieval Chain

Connect vector retrieval and DeepSeek model to build a complete RAG QA chain. LangChain's RetrievalQA chain encapsulates the entire process from retrieval to generation.

Build RetrievalQA Chain

from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate

# Initialize DeepSeek model (via OpenAI-compatible interface)
llm = ChatOpenAI(
    model="deepseek-chat",
    base_url="https://api.deepseek.com/v1",
    api_key="sk-your-api-key-here",
    temperature=0.3,           # RAG scenario recommends low temperature for accuracy
    max_tokens=2048,
)

# Custom prompt template
prompt_template = """You are a professional QA assistant based on a knowledge base. Please answer user questions based on the following document content.

Requirements:
1. Answer only based on the provided document content, do not make up information
2. If the document content is insufficient to answer the question, clearly state "Based on the existing documents, this question cannot be answered"
3. Cite specific document sources in your answer
4. Answer in Chinese, keep it professional and clear

Document content:
{context}

User question: {question}

Answer: """

PROMPT = PromptTemplate(
    template=prompt_template,
    input_variables=["context", "question"],
)

# Build RetrievalQA chain
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",            # Put all retrieval results into prompt
    retriever=vectorstore.as_retriever(
        search_kwargs={"k": 4}     # Retrieve 4 most relevant documents
    ),
    chain_type_kwargs={"prompt": PROMPT},
    return_source_documents=True,     # Return source documents
)

Test QA

# Ask question and get answer
question = "What is the pricing of DeepSeek API?"
result = qa_chain.invoke({"query": question})

print("Question:", question)
print("Answer:", result["result"])
print("\nSources:")
for i, doc in enumerate(result["source_documents"]):
    source = doc.metadata.get("source", "Unknown")


print(f"  {i+1}. {source}")

chain_type Parameter Description

Type	Description	Use Case
stuff	Put all retrieval results into the prompt at once	Few documents (k <= 4), most common
map_reduce	Summarize each document individually, then combine summaries	Many and long documents, need global understanding
refine	Iteratively refine the answer document by document	Need high-quality answers, don't mind latency
map_rerank	Score each document, select the highest-scoring answer	Need precise matching of specific documents

Step 8

Q&A with Source Citations

One of the core advantages of RAG is traceability. Every answer can be traced back to specific document sources, allowing users to verify the accuracy of information. Below shows how to present Q&A results with source citations in the UI.

Formatting Source Citations

def format_qa_response(result):
    """Format RAG Q&A result with source citations"""
    answer = result["result"]
    sources = result["source_documents"]

    # Extract unique sources
    unique_sources = []
    seen = set()
    for doc in sources:
        source = doc.metadata.get("source", "Unknown source")
        page = doc.metadata.get("page", None)
        source_key = f"{source}_{page}"
        if source_key not in seen:
            seen.add(source_key)
            unique_sources.append({
                "source": source,
                "page": page,
                "preview": doc.page_content[:150] + "..."
            })

    return {
        "answer": answer,
        "sources": unique_sources,
        "source_count": len(unique_sources),
    }

# Usage example
result = qa_chain.invoke({"query": "What are the features of the DeepSeek R1 model?"})
formatted = format_qa_response(result)

print("=" * 60)
print(formatted["answer"])
print("\n--- References (", formatted["source_count"], ") ---")
for i, s in enumerate(formatted["sources"]):
    page_info = f"Page {s['page']}" if s['page'] else ""
print(f"{i+1}. {s['source']} {page_info}")
    print(f"   {s['preview']}")

Source Citation Format in Prompt

In the prompt, requiring the model to annotate source numbers when citing can effectively improve traceability:

# Prompt template with numbered citations
prompt_with_citation = """You are a professional Q&A assistant based on a knowledge base. Please answer user questions based on the following document content.

Requirements:
1. Answer only based on the provided document content, do not fabricate information
2. Cite specific sources in the answer, format as [Source X]
3. If the document content is insufficient to answer the question, clearly state so
4. Answer in Chinese, keep it professional and clear

Document content:
{context}

User question: {question}

Please answer (cite sources with [Source 1], [Source 2], etc.):"""

Building a Web Q&A API

Use FastAPI to wrap the RAG system as a Web API:

# pip install fastapi uvicorn
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title="DeepSeek RAG API")

class Question(BaseModel):
    text: str

class Answer(BaseModel):
    question: str
    answer: str
    sources: list

@app.post("/ask", response_model=Answer)
async def ask_question(q: Question):
    result = qa_chain.invoke({"query": q.text})
    formatted = format_qa_response(result)
    return Answer(
        question=q.text,
        answer=formatted["answer"],
        sources=[s["source"] for s in formatted["sources"]],
    )

# Start service: uvicorn main:app --reload --port 8000

Step 9

Advanced RAG Techniques

Basic RAG can solve 80% of scenarios, but complex scenarios require more advanced retrieval strategies. The following techniques can help you significantly improve the answer quality of your RAG system.

Multi-Query Retrieval

Use the LLM to automatically rewrite the user's question into multiple queries from different angles, retrieve them separately, and merge the results to improve recall:

from langchain.retrievers.multi_query import MultiQueryRetriever

# Create a multi-query retriever
multi_query_retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(),
    llm=llm,
)

# User question: "How to deploy the model?"
# Automatically generate multiple queries:
# 1. "DeepSeek model local deployment method"
# 2. "Ollama deployment DeepSeek tutorial"
# 3. "DeepSeek model server deployment steps"

unique_docs = multi_query_retriever.invoke("How to deploy the model?")
print(f"Multi-query retrieved {len(unique_docs)} relevant documents")

Parent Document Retriever

Use small text chunks for retrieval (to improve precision), but return large text chunks (to preserve context). This solves the problem of losing context with small chunks:

from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore

# Create two splitters: small chunks for child documents, large chunks for parent documents
child_splitter = RecursiveCharacterTextSplitter(chunk_size=200)
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=1000)

# Parent document retriever
parent_retriever = ParentDocumentRetriever(
    vectorstore=Chroma(
        collection_name="parent_docs",
        embedding_function=embedding_model,
    ),
    docstore=InMemoryStore(),
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)

parent_retriever.add_documents(documents)
# Retrieve using child documents, but return the full parent document
results = parent_retriever.invoke("DeepSeek API pricing")

Contextual Compression

After retrieving documents, use a compressor to extract the most relevant parts to the question, removing redundant information:

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

# Create a compressor: use LLM to extract parts of documents relevant to the question
compressor = LLMChainExtractor.from_llm(llm)

compression_retriever = ContextualCompressionRetriever(
base_compressor=compressor,
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 6}),
)

# Retrieve and compress documents
compressed_docs = compression_retriever.invoke(
    "What is the difference between DeepSeek V3 and R1?"
)
for doc in compressed_docs:
    print(f"Compressed content length: {len(doc.page_content)} characters")

Re-ranking

First use vector retrieval to get candidate documents, then use a re-ranking model to re-rank the candidate documents, greatly improving retrieval precision:

# pip install sentence-transformers
from sentence_transformers import CrossEncoder

# Load re-ranking model
reranker = CrossEncoder("BAAI/bge-reranker-large")

# Step 1: Vector retrieval to get candidate documents (retrieve more)
candidate_docs = vectorstore.similarity_search(query, k=20)

# Step 2: Use re-ranking model to re-score
pairs = [[query, doc.page_content] for doc in candidate_docs]
scores = reranker.predict(pairs)

# Step 3: Sort by score, take top K
sorted_docs = sorted(
    zip(candidate_docs, scores),
    key=lambda x: x[1],
    reverse=True
)
top_docs = [doc for doc, score in sorted_docs[:4]]

print(f"After re-ranking, retained {len(top_docs)} most relevant documents")

Summary of Advanced Techniques

Technique	Problem Solved	Cost
Multi-Query Retrieval	Retrieval failure due to imprecise user question phrasing	Additional LLM calls
Parent Document Retrieval	Small chunks lose context	Storage doubles
Contextual Compression	The retrieval results contain a lot of irrelevant content	Additional LLM calls
Re-ranking	Vector retrieval is not precise enough	Additional model inference

Step 10

Complete Hands-on Project

Integrate all the previous steps into a complete document Q&A system: load PDF folders, create a vector database, interactive Q&A, and output with source citations.

Complete Code: deepseek_rag.py

"""DeepSeek RAG Knowledge Base Q&A System — Complete Implementation"""
import os
from langchain_community.document_loaders import DirectoryLoader, PyPDFLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate

# ========== Configuration ==========
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY", "sk-your-api-key")
DOCS_DIR = "./documents"
CHROMA_DIR = "./chroma_db"
EMBEDDING_MODEL = "BAAI/bge-small-zh-v1.5"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 100
RETRIEVAL_K = 4

# ========== 1. Load Documents ==========
def load_documents(docs_dir):
    """Load all PDF and TXT files under docs_dir"""
    loaders = {
        "**/*.pdf": PyPDFLoader,
        "**/*.txt": TextLoader,
    }
    all_docs = []
    for pattern, loader_cls in loaders.items():
        loader = DirectoryLoader(docs_dir, glob=pattern, loader_cls=loader_cls)
        try:
            docs = loader.load()
            all_docs.extend(docs)
            print(f"  [OK] {pattern}: {len(docs)} documents")
        except Exception as e:
            print(f"  [SKIP] {pattern}: {e}")
    return all_docs



# ========== 2. Split documents ==========
def split_documents(documents):
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=CHUNK_SIZE,
        chunk_overlap=CHUNK_OVERLAP,
        separators=["\n\n", "\n", "。", "！", "？", ".", " ", ""],
    )
    chunks = splitter.split_documents(documents)
    print(f"  Original documents: {len(documents)}, after split: {len(chunks)} chunks")
    return chunks

# ========== 3. Create vector database ==========
def create_vectorstore(chunks, force_rebuild=False):
    embedding = HuggingFaceEmbeddings(
        model_name=EMBEDDING_MODEL,
        model_kwargs={"device": "cpu"},
        encode_kwargs={"normalize_embeddings": True},
    )

    if os.path.exists(CHROMA_DIR) and not force_rebuild:
        print("  Loading existing vector database...")
        vectorstore = Chroma(
            persist_directory=CHROMA_DIR,
            embedding_function=embedding,
        )
    else:
        print("  Creating new vector database...")
        vectorstore = Chroma.from_documents(
            documents=chunks,
            embedding=embedding,
            persist_directory=CHROMA_DIR,
        )

    print(f"  Vector database: {vectorstore._collection.count()} records")
    return vectorstore

# ========== 4. Build QA chain ==========
def create_qa_chain(vectorstore):
    llm = ChatOpenAI(
        model="deepseek-chat",
        base_url="https://api.deepseek.com/v1",
        api_key=DEEPSEEK_API_KEY,
        temperature=0.3,
        max_tokens=2048,
    )

    prompt = PromptTemplate(
        template="""You are a professional Q&A assistant based on the knowledge base. Please answer the user's question based on the following document content.

Requirements:
1. Answer only based on the provided document content, do not make up information
2. If the document content is insufficient to answer the question, please state clearly
3. Cite the document source in the answer, format as [Source X]
4. Answer in Chinese, keep professional and clear

Document content:
{context}

User question: {question}

Answer: """,
        input_variables=["context", "question"],
    )

    qa_chain = RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff",
retriever=vectorstore.as_retriever(
            search_kwargs={"k": RETRIEVAL_K}
        ),
        chain_type_kwargs={"prompt": prompt},
        return_source_documents=True,
    )
    return qa_chain

# ========== 5. Interactive Q&A ==========
def interactive_qa(qa_chain):
    print("\n" + "=" * 60)
    print("  DeepSeek RAG Knowledge Base Q&A System")
    print("  Enter 'quit' to exit, 'sources' to show last sources")
    print("=" * 60)

    last_result = None

    while True:
        try:
            question = input("\nYour question: ").strip()
        except (EOFError, KeyboardInterrupt):
            break

        if not question:
            continue
        if question.lower() == "quit":
            break
        if question.lower() == "sources" and last_result:
            print("\n--- Reference sources for last answer ---")
            for i, doc in enumerate(last_result["source_documents"]):
                source = doc.metadata.get("source", "Unknown")
                print(f"  [Source{i+1}] {source}")
            continue

        print("  Thinking...", end="\r")
        result = qa_chain.invoke({"query": question})
        last_result = result

        print("\n" + "-" * 60)
        print(result["result"])
        print("-" * 60)

        # Display source summary
        sources = set()

        for doc in result["source_documents"]:
            sources.add(doc.metadata.get("source", "unknown"))
        print(f"Sources ({len(sources)}): {', '.join(list(sources)[:3])}")

    print("\n  Goodbye!")

# ========== Main Program ==========
if __name__ == "__main__":
    print("[1/4] Loading documents...")
    documents = load_documents(DOCS_DIR)

    if not documents:
        print("  Error: No documents found. Please place PDF/TXT files in the ./documents directory")
        exit(1)

    print("\n[2/4] Splitting documents...")
    chunks = split_documents(documents)

    print("\n[3/4] Creating vector database...")
    vectorstore = create_vectorstore(chunks)

    print("\n[4/4] Building QA chain...")
    qa_chain = create_qa_chain(vectorstore)

    interactive_qa(qa_chain)

Run the Project

# 1. Create documents directory
mkdir documents

# 2. Put your PDF/TXT files
# Place enterprise documents, technical manuals, etc. into the documents/ directory

# 3. Set API Key
# Windows:
# $env:DEEPSEEK_API_KEY = "sk-your-api-key"
# macOS/Linux:
# export DEEPSEEK_API_KEY="sk-your-api-key"

# 4. Run
python deepseek_rag.py

Step 11

Integration with Dify

If you don't want to write code, the Dify platform provides a visual RAG knowledge base feature. Apply the above RAG concepts to Dify to build an enterprise-level knowledge base Q&A system without coding.

RAG Mapping in Dify

RAG Stage	Code Implementation	Dify Equivalent
Document Loading	PyPDFLoader / TextLoader	Knowledge Base → Upload Documents (supports PDF/Word/TXT/Web)
Text Splitting	RecursiveCharacterTextSplitter	Knowledge Base → Chunking Settings (custom chunk length and overlap)
Vector Embedding	HuggingFaceEmbeddings	Model Providers → Embedding Model Configuration
Vector Storage	ChromaDB	Dify Built-in Vector Database (Qdrant/Weaviate/ChromaDB)
Retrieval	vectorstore.similarity_search()	Knowledge Base → Recall Settings (TopK, Score Threshold)
Generation	ChatOpenAI + RetrievalQA	Application Orchestration → Add DeepSeek Model + Knowledge Base Node

Dify Configuration Steps

Deploy Dify: Use Docker Compose for one-click deployment (see DeepSeek Ecosystem Tools Dify section)
Configure Model: Add DeepSeek in "Settings → Model Providers" (OpenAI-API-compatible method)
Create Knowledge Base: Upload documents, set chunking parameters (chunk_size 500, chunk_overlap 100)
Build Application: Create a "Chat Application" or "Agent", add knowledge base node in orchestration canvas
Publish and Use: One-click publish as Web App or API, supports embedding into existing systems

Advantages of Dify

Visual operation, no coding required
Built-in vector database, automatic index management
Supports hybrid retrieval (vector retrieval + keyword retrieval)
Built-in citation tracing, automatically annotates answer sources
Conversation logs and analytics, continuously optimize Q&A performance
One-click publish as API, easy integration into business systems

For more Dify tips and DeepSeek integration solutions, see DeepSeek Ecosystem Tools and DeepSeek Deployment Tutorial.

RAG Help

DeepSeek RAG Knowledge Base FAQ

RAG or Fine-tuning: Which should I choose? +

RAG is suitable for dynamic knowledge scenarios (frequently updated knowledge, need for traceability, limited budget), while fine-tuning is suitable for static capability scenarios (style imitation, specific terminology, instruction following). For most enterprise scenarios, a combination with RAG as the primary approach and fine-tuning as a supplement is recommended. RAG is cost-effective and delivers quick results, so it's advisable to start with RAG.

How should I set chunk_size and chunk_overlap? +

For chunk_size, 500-1000 characters is recommended, and chunk_overlap should be 10%-20% of chunk_size. Use 500 for technical documents, 800-1000 for long articles, and 300-500 for FAQ-type content. chunk_overlap cannot be omitted; it ensures key information is not lost at chunk boundaries. Start with default values for testing, then fine-tune based on retrieval performance.

What embedding model is recommended for Chinese documents? +

We strongly recommend BAAI/bge-small-zh-v1.5 (96MB, lightweight and efficient) and BAAI/bge-large-zh-v1.5 (1.3GB, highest accuracy). These two models significantly outperform general English models in Chinese semantic understanding. If using the Dify platform, you can directly configure these embedding models in the model provider. For local deployment, load them with HuggingFaceEmbeddings.

How to choose between ChromaDB and FAISS? +

ChromaDB is suitable for small to medium scale (tens of thousands to hundreds of thousands of vectors), with automatic data persistence to disk and easy usage. FAISS is suitable for large scale (millions and above), with extremely fast in-memory operations, but requires manual persistence management. For personal projects or small teams, ChromaDB is sufficient; for large-scale production data, use Milvus or Qdrant.

What should I do if retrieval results are inaccurate? +

1) Adjust chunk_size and chunk_overlap parameters; 2) Use Multi-Query retrieval to automatically rewrite user questions; 3) Add Re-ranking to re-rank retrieval results; 4) Optimize the Prompt template to explicitly require the model to answer based on documents; 5) Use hybrid retrieval (vector retrieval + keyword retrieval). It is recommended to start by adjusting chunk parameters, as this has the most noticeable effect.

Can I use locally deployed DeepSeek with Ollama for RAG? +

Absolutely. Simply change the base_url of ChatOpenAI to http://localhost:11434/v1, set model to deepseek-r1:8b (or the model name you use), and set api_key to any non-empty string. You can also use local HuggingFaceEmbeddings for the embedding model, allowing the entire RAG system to run completely offline with no data leaving your environment. See the DeepSeek Local Deployment Tutorial for details.

Learn More

DeepSeek Related Tutorials

In-depth learning on using, deploying, and ecosystem tools for DeepSeek models.

Beginner Tutorial

How to Use DeepSeek Models

Four usage methods, zero-basics beginner tutorial.

Deployment Tutorial

DeepSeek Deployment Tutorial

Deployment solutions with Ollama, Docker, vLLM, K8s.

Ecosystem Tools

DeepSeek Ecosystem Tools

WebUI, IDE plugins, Agent frameworks, RAG platforms.

Model Details

DeepSeek Model Architecture

Technical architecture, Benchmark, selection comparison.

Model List

DeepSeek Open Source Models

Complete catalog of 6 series, 20+ models.

Download Guide

DeepSeek Model Download

Download guides for Ollama, Hugging Face, GitHub.

DeepSeek RAG Knowledge Base Setup Tutorial

What is RAG and why do we need it?

RAG Principle Overview

The Three Core Stages of RAG

RAG Architecture Flow

RAG vs Fine-tuning

Environment Preparation

Install Core Dependencies

DeepSeek API Configuration

Document Loading

Loading PDF Documents

Loading TXT and Markdown

Loading Web Pages

Using Unstructured to Load General Documents

Text Splitting

RecursiveCharacterTextSplitter Core Parameters

Text Splitting Code

Best Practices for Text Splitting

Best Practices for Text Splitting

Vector Embeddings

Embedding Model Selection

Vector Embedding Code

Vector Database — ChromaDB

Create Vector Database and Store

Load Existing Vector Database

Similarity Search

Other Vector Database Options

Build Retrieval Chain

Build RetrievalQA Chain

Test QA

chain_type Parameter Description

Q&A with Source Citations

Formatting Source Citations

Source Citation Format in Prompt

Building a Web Q&A API

Advanced RAG Techniques

Multi-Query Retrieval

Parent Document Retriever

Contextual Compression

Re-ranking

Summary of Advanced Techniques

Complete Hands-on Project

Complete Code: deepseek_rag.py

Run the Project

Integration with Dify

RAG Mapping in Dify

Dify Configuration Steps

DeepSeek RAG Knowledge Base FAQ

DeepSeek Related Tutorials

How to Use DeepSeek Models

DeepSeek Deployment Tutorial

DeepSeek Ecosystem Tools

DeepSeek Model Architecture

DeepSeek Open Source Models

DeepSeek Model Download

每日精选 Skill 推荐，免费送到你邮箱