Chapter 10 · Chain-of-Thought

Chapter goals: Understand the principle and variants of Chain-of-Thought (CoT) prompting, master Manual-CoT, Zero-Shot-CoT, and Auto-CoT usage, and learn to judge when to use CoT and when not to.

10.1 Why We Need Chain-of-Thought

Standard prompts make the model jump directly from question to answer. This is fine for simple tasks, but when facing multi-step reasoning (math word problems, logical deduction, commonsense推理), the model is prone to errors in intermediate steps, leading to wrong final answers.

The core idea of CoT: have the model explicitly write out intermediate reasoning steps instead of directly giving the final answer.

# Standard prompt — ask for answer directly
prompt = "Xiao Ming has 5 apples, gives 2 to Xiao Hong, then buys 3 more. How many apples does Xiao Ming have?"

# CoT prompt — ask to show the reasoning process
prompt = """Xiao Ming has 5 apples, gives 2 to Xiao Hong, then buys 3 more. How many apples does Xiao Ming have?
Please think step by step and write out your reasoning."""

Wei et al. (2022) first systematically validated this method in their paper Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Experiments showed that on the GSM8K math benchmark, PaLM 540B with CoT prompting improved accuracy from 17.9% to 56.9% — nearly a threefold increase.

Applicable Conditions

CoT is a reasoning enhancement technique, not a silver bullet. It is most effective for:

• Math computation and logical reasoning
• Multi-step planning
• Commonsense causal reasoning

For simple classification, sentiment analysis, or factual retrieval tasks, CoT may instead introduce unnecessary noise.

10.2 Three CoT Variants

Manual-CoT (Hand-Crafted Examples)

Manually write an example containing a full reasoning process in the prompt, letting the model imitate:

prompt = """
Q: There are 15 fish in a pond. 3 die. How many living fish remain?
A: Let's think step by step. There were originally 15 fish in the pond. 3 dying doesn't mean they left the pond,
   so the total number of fish in the pond remains unchanged. The question asks "how many living fish remain" — 15 - 3 = 12.
   The answer is 12 living fish.

Q: A store offers a 20% discount on an item originally priced at $80. What is the current price?
A:
"""

Pros: highest controllability; example quality directly affects output quality. Cons: high manual design cost.

Zero-Shot-CoT (Zero-Shot Chain-of-Thought)

Kojima et al. (2022) found that simply appending the magic phrase "Let's think step by step" at the end of a prompt activates the model's step-by-step reasoning capability:

prompt = """A store offers a 20% discount on an item originally priced at $80. What is the current price?

Let's think step by step."""

Chinese equivalent: "请一步一步思考" or "让我们逐步分析".

import requests

resp = requests.post(
    "https://api.deepseek.com/v1/chat/completions",
    headers={"Authorization": "Bearer <key>"},
    json={
        "model": "deepseek-chat",
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0
    }
)
print(resp.json()["choices"][0]["message"]["content"])

Pros: no need to design examples, ready to use out of the box. Cons: effectiveness depends on model size — adding it to small models (<10B) does nothing.

Few-Shot-CoT (Few-Shot Chain-of-Thought)

Combine both: provide several examples with full reasoning chains, letting the model learn the reasoning pattern before solving new problems. This is the most effective approach.

10.3 Auto-CoT: Automatically Building Reasoning Chains

Zhang et al. (2023) proposed Auto-CoT, replacing manual example design with a two-step automation:

1. Clustering sampling: cluster the question set by semantic vectors, pick a representative question from each cluster;
2. Automatic reasoning chain generation: use Zero-Shot-CoT to generate reasoning steps for each representative question, serving as few-shot examples.

from sklearn.cluster import KMeans
from sentence_transformers import SentenceTransformer

# 1. Vectorize the question set
model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = model.encode(questions)

# 2. Cluster
kmeans = KMeans(n_clusters=5)
clusters = kmeans.fit_predict(embeddings)

# 3. Pick one representative question per cluster, generate CoT examples
for cluster_id in range(5):
    rep_question = select_representative(questions, clusters, cluster_id)
    cot_example = generate_cot(rep_question)  # using ZS-CoT
    examples.append(cot_example)

10.4 When to Use / Not to Use CoT

Scenario	Recommendation	Reason
Math / logical reasoning	✅ Strongly recommended	Intermediate steps significantly reduce error rate
Multi-step planning	✅ Recommended	Decomposing tasks reduces omissions
Simple classification	❌ Not recommended	Wastes tokens for no gain
Creative writing	⚠️ Depends	May constrain creativity
Factual verification	❌ Not recommended	Reasoning chains can't compensate for knowledge gaps

Cost consideration: CoT increases output token count (the reasoning chain itself takes up space). In high-concurrency API scenarios, weigh latency against cost.

10.5 Relationship to Later Chapters

• Chapter 11's Self-Consistency is an enhanced version of CoT: multiple sampling + voting;
• Chapter 12's ReAct adds action capability (tool calling) on top of CoT;
• Chapter 13's ToT upgrades the linear reasoning chain to tree-shaped exploration.

The three build progressively; CoT is the cornerstone of all reasoning enhancement techniques.

Chapter Summary

• CoT makes the model output intermediate reasoning steps, dramatically improving accuracy on complex reasoning;
• Three variants: Manual-CoT (hand-crafted examples), Zero-Shot-CoT (magic phrase), Few-Shot-CoT (few-shot + reasoning chain);
• Auto-CoT reduces manual cost via clustering + automatic generation;
• Don't滥用 CoT on simple classification tasks — it only adds cost without improving results.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the core idea of Chain-of-Thought?

2. Which phrase does Zero-Shot-CoT need to be appended to a prompt to take effect?

3. Which of the following tasks is most suitable for using CoT?

4. What are the two steps of Auto-CoT for automatically building reasoning chains?

🛠️ Hands-on Practice

1. Use Zero-Shot-CoT to solve a math word problem (e.g. chickens and rabbits in the same cage); compare results with and without "Let's think step by step."
2. Manually write a Few-Shot-CoT example (with full reasoning chain) to solve a logical reasoning problem.
3. Test whether adding CoT to your model improves performance on a small-scale sentiment classification task — record the token consumption comparison between the two approaches.