Core Challenges of Multi-Turn Dialogue
Unlike single-turn Q&A, multi-turn dialogue faces three core challenges: limited context window (even with 128K models, long conversations gradually exceed the window), dialogue state tracking (the AI needs to remember what the user said earlier and what decisions were made), and topic drift management (users may suddenly switch topics, and the AI needs to respond flexibly). These challenges compound, making it much harder to build a high-quality multi-turn dialogue system than one might imagine.
Many people think that simply concatenating all historical messages and sending them to the model can achieve multi-turn dialogue. This works for short conversations, but as the dialogue grows, problems gradually emerge: token consumption increases linearly, the model's attention is diluted by irrelevant history, response speed slows down, and costs keep rising. According to statistics, in conversations exceeding 20 turns, the information from the first 5 turns contributes less than 5% to the current reply, yet they consume over 60% of the context tokens.
Fine-Grained Management of the Context Window
The core principle of context window management is: don't stuff all history into the model at once; instead, strategically select the most valuable context. Common strategies include: sliding window (keeping only the most recent N turns), key information summarization (compressing early dialogue into a summary), and hybrid strategies (keeping the original text for the recent N turns, and replacing earlier ones with summaries).
The sliding window strategy is simple to implement but loses early key information—for example, if the user mentions their budget in turn 1, by turn 15 when asking for recommendations, the AI has already forgotten the budget. The summarization strategy can preserve key information but may lose details. The best practice is a combination of both: compress history beyond a threshold into a summary, keep the full original text for the last 10 turns, and in the summary, emphasize structural information such as user profile, key decisions, and pending tasks.
from openai import OpenAI
client = OpenAI(api_key="your-deepseek-api-key", base_url="https://api.deepseek.com")
class ContextManager:
def __init__(self, max_raw_turns=8, max_total_tokens=8000):
self.max_raw_turns = max_raw_turns
self.summary = ""
self.raw_history = []
self.user_profile = {}
def add_turn(self, role, content):
self.raw_history.append({"role": role, "content": content})
if role == "user":
self._update_profile(content)
if len(self.raw_history) > self.max_raw_turns * 2:
self._compress()
def _update_profile(self, content):
response = client.chat.completions.create(
model="deepseek-chat",
messages=[{"role":"system","content":"Extract key information from user message as JSON: {preferences:[],constraints:[],decisions:[]}"},
{"role":"user","content":content}], temperature=0.1)
def _compress(self):
old_turns = self.raw_history[:-self.max_raw_turns*2]
self.raw_history = self.raw_history[-self.max_raw_turns*2:]
history_text = "\n".join(f"{t['role']}: {t['content'][:200]}" for t in old_turns)
response = client.chat.completions.create(
model="deepseek-chat",
messages=[{"role":"system","content":"Compress into a 100-word summary, preserving user needs, decisions, and unresolved issues"},
{"role":"user","content":history_text}], temperature=0.1)
self.summary = response.choices[0].message.content
def build_messages(self, system_prompt):
msgs = [{"role":"system","content":system_prompt}]
if self.summary:
msgs.append({"role":"system","content":f"[History Summary] {self.summary}\n[User Profile] {self.user_profile}"})
msgs.extend(self.raw_history)
return msgs
ctx = ContextManager(max_raw_turns=6)
ctx.add_turn("user","I want to buy a laptop around 5000 yuan, mainly for programming")
ctx.add_turn("assistant","Okay, for a 5000 yuan programming laptop, I recommend ThinkBook 14+...")
msgs = ctx.build_messages("You are a computer shopping expert.")Dialogue State Tracking
Dialogue State Tracking (DST) is a core component of multi-turn dialogue systems. Its task is to maintain a structured dialogue state—including the user's intent, provided slot information (such as departure, destination, date), and the current stage of the dialogue. Good state tracking allows the AI to always know where the conversation is.
There are multiple ways to implement DST: the simplest is to maintain a state block in the system prompt and update it each turn; medium complexity is to store state in a JSON structure and update it via Function Calling; the most complex but most reliable is to maintain a state machine at the code level and decide prompt strategies based on the state. For most applications, the JSON state block approach achieves a good balance between effectiveness and complexity.
class DialogueStateTracker:
def __init__(self):
self.state = {"phase":"greeting","intent":None,"slots":{},"missing_slots":[],"turn_count":0}
def update(self, user_msg, asst_msg):
self.state["turn_count"] += 1
response = client.chat.completions.create(
Topic Switching and Drift Handling
In real conversations, users often suddenly switch topics—"Oh, by the way..." "Let's not talk about that, let's talk about...". AI needs to handle such topic switches gracefully without losing previous context. Key strategies include: recognizing the switch intent (whether it's a temporary question or a permanent switch), preserving the old topic state (so that when the user returns, it can pick up where it left off), and confirming the switch (for important topic changes, confirm with the user: "Would you like to address XX first, or YY?"). Topic switch detection can be implemented via model judgment or rule-based matching. Simple rules: detect switch keywords ("by the way", "change the subject", "let's not talk about this"), combined with semantic similarity calculation (a sudden increase in semantic distance between the current message and the previous turn).
Long-term Memory and Personalization
Cross-session memory is the ultimate challenge in multi-turn dialogue. The user chats today, returns tomorrow—can the AI still remember? This requires a persistent user memory system. Common implementation approaches: vector-based semantic memory (encode each conversation into a vector and store it in a database; when a new conversation occurs, retrieve the most relevant history), structured profiles (extract and store user preferences, habits, important decisions), and conversation summary chains (automatically generate a summary after each session and archive it).
class LongTermMemory:
def __init__(self):
self.sessions = {}
self.user_profile = {"preferences":{},"expertise_level":"unknown","common_topics":[],"interaction_style":"unknown"}
def summarize_session(self, session_id, messages):
dialogue = "\n".join(f"{m['role']}: {m['content'][:300]}" for m in messages)
response = client.chat.completions.create(
model="deepseek-chat",
messages=[{"role":"system","content":"Extract key information from the dialogue: 1. User goal 2. Conclusions reached 3. Unresolved issues 4. User preferences. Within 150 characters."},
{"role":"user","content":dialogue}])
self.sessions[session_id] = response.choices[0].message.content
def recall_relevant(self, current_query):
relevant = [f"Session {sid}: {summary}" for sid, summary in list(self.sessions.items())[-5:]]
return "\n".join(relevant) if relevant else "No relevant history"
memory = LongTermMemory()
memory.summarize_session("s001",[{"role":"user","content":"Help me analyze competitor A's pricing strategy"},{"role":"assistant","content":"Competitor A uses a three-tier pricing..."}])The Trilemma of Cost Optimization
Multi-turn dialogue systems face the "quality-cost-latency" impossible triangle: to improve quality, you need to pass more context (increasing cost and latency); to reduce cost, you need to compress context (possibly reducing quality); to reduce latency, you need to simplify processing (also affecting quality). In real projects, you need to find the optimal balance among these three based on the business scenario. For customer service systems: latency is more important than cost, but context management can be aggressive; for AI tutoring systems: quality is most important, cost can be relaxed; for entertainment chatbots: cost is most important, and aggressive compression strategies can be used.
Production Deployment Checklist
- Context Limit Protection: Set a hard token limit (e.g., not exceeding 70% of the model's context window) to prevent truncation or errors due to exceeding the window.
- Conversation Timeout and Cleanup: Set a session timeout (e.g., automatically end after 30 minutes of inactivity) and clean up zombie sessions to free memory.
- Concurrent Session Isolation: Ensure that session states of different users are completely isolated to prevent information leakage due to shared memory.
- Degradation Strategy: When the model returns abnormal results in a turn (e.g., overly long output, format errors), have a fallback mechanism to ensure the conversation does not break.
- Monitoring and Alerting: Monitor key metrics such as average number of dialogue turns, user satisfaction, abnormal exit rate, etc.
Want to orchestrate this skill chain yourself?
Open in Skill Chain →