Overview of Chain-of-Thought Prompting

Chain-of-Thought (CoT) is a prompting technique that significantly improves model performance on complex reasoning tasks by guiding the model to show intermediate reasoning steps before giving the final answer. Introduced by Google Research in 2022, this technique has proven effective in tasks such as mathematical reasoning, logical reasoning, and commonsense reasoning.

How CoT Works

Traditional prompting directly asks the model for an answer, while CoT prompting requires the model to "think step by step." For example:

# Traditional Prompt
Question: Xiao Ming has 5 apples, gives 2 to Xiao Hong, and buys 3 more. How many does he have now?
Answer: 6

# CoT Prompt
Question: Xiao Ming has 5 apples, gives 2 to Xiao Hong, and buys 3 more. How many does he have now?
Let's think step by step:
1. Xiao Ming initially has 5 apples
2. Gives 2 to Xiao Hong, remaining: 5 - 2 = 3
3. Buys 3 more, now: 3 + 3 = 6
Answer: 6

Zero-shot CoT

The simplest form of CoT is Zero-shot CoT, which only requires adding "Let's think step by step" at the end of the prompt to trigger the model's reasoning chain. This method does not require any examples, but its performance is usually not as good as Few-shot CoT.

Few-shot CoT

Few-shot CoT guides the model to imitate reasoning patterns by providing 2-3 examples with complete reasoning processes. This is currently the most effective CoT method:

Please answer the question following the reasoning style of the examples below.

Example 1:
Question: A class has 30 students, 60% of whom are girls. One-third of the girls wear glasses. How many girls wear glasses?
Reasoning:
- Number of girls: 30 × 60% = 18
- Girls wearing glasses: 18 × 1/3 = 6
Answer: 6

Example 2:
Question: A store is having a promotion. An item originally priced at 200 yuan is discounted by 20% and then reduced by 20 yuan. What is the final price?
Reasoning:
- After 20% discount: 200 × 0.8 = 160 yuan
- Then reduced by 20 yuan: 160 - 20 = 140 yuan
Answer: 140 yuan

Now answer:
Question: A book originally costs 45 yuan. After a 30% discount, Xiao Ming buys two books with 100 yuan. How much change should he get?

Few-shot Prompting Tips

Few-shot prompting is not only applicable to CoT but is also an important technique in general prompt engineering:

  • Quality over quantity: 2-3 high-quality examples are often better than 5 mediocre ones
  • Example diversity: Examples covering different scenarios help the model generalize better
  • Format consistency: Keep the example format consistent with the expected output format
  • Label key information: Use labels to mark input and output in examples

Scenarios Where CoT is Effective

CoT is particularly effective in the following scenarios:

  • Mathematical and arithmetic reasoning
  • Logical reasoning and puzzle solving
  • Multi-step problem solving
  • Code debugging and explanation
  • Commonsense reasoning

Note that CoT is not very helpful for simple tasks (such as direct factual Q&A) and may even add unnecessary reasoning cost.

Best Practices Summary

1. For complex reasoning tasks, prefer Few-shot CoT. 2. For simple tasks, use Zero-shot CoT or direct prompting. 3. Ensure the reasoning in examples is correct and clear. 4. In API calls, CoT increases token consumption, so weigh cost against effectiveness.