Chain-of-Thought and ReAct

Wei et al., 2022 · Yao et al., 2022

Tokens are where the thinking happens. Not a metaphor — a budget.

Download chain_of_thought_from_scratch.py — everything below runs from it. NumPy and plain Python. There is no language model anywhere in it: what the script runs is the mechanism — the per-token compute budget, the loop shape, and the voting.

Blunt about that last bit first, the same way the RAG page is about its retriever. There is no LLM in this script. Nothing it prints is evidence about what GPT-4 does when you say "think step by step"; where something stands in for a model, it's a hand-written state machine and the code says so on the line above.

What it does show is the part I had backwards for two years. I thought chain-of-thought was a psychological trick — flatter the model, make it try harder. It isn't. It's an allocation of compute. Once I saw that, the rest of the agent stack stopped being prompt folklore and became something I could reason about like any other system.

A pure function versus one with a scratch buffer

You've written this function. It has no local variables:

def answer(question):
    return LOOKUP_TABLE[question]     # one step. That's all you get.

A transformer emitting one token is that function. One forward pass, a fixed number of layers, the same arithmetic whether the question is "what colour is the sky" or a 12-step word problem. It cannot loop internally. There is no while in there.

Chain-of-thought gives it somewhere to put the intermediate values:

def answer(question):
    scratch = []                      # each entry is another forward pass
    for step in range(n):
        scratch.append(one_step(question, scratch))   # reads its own output
    return scratch[-1]

Every token the model emits gets appended to its own input, and the next forward pass reads it. So the number of tokens you let it emit is the number of serial steps it gets. Nothing else about the model changed.

TermMeansProgrammer's analogy
Forward passFixed work per token: N layers, no loopsA function with a fixed call depth
Chain-of-thoughtEmit intermediate tokens before the answerA scratch buffer the function can write to and re-read
Self-consistencySample several paths, take the modal answerRetry the flaky job 20 times, ship the most common result
ReActLet the scratch buffer hold a real tool's outputThe loop body makes an API call instead of guessing the response

Why this beat plain few-shot prompting

Standard few-shot prompting, the thing GPT-3 introduced, shows the model input/output pairs and asks for one more output. It works beautifully for classification and translation, and it falls over on arithmetic word problems in a way that looked, for a while, like the models simply couldn't do maths.

They could. They were being asked to do six sequential operations inside one forward pass. The fix wasn't a bigger model or a new loss — it was changing the exemplars so they showed their working, and therefore so did the continuation. No fine-tuning, no gradient updates, no architecture change.

The one formula

Chain-of-thought doesn't really have a formula; it's a prompt format. But self-consistency does, and it's the one worth knowing, because it says what a "reasoning path" is in probability terms:

answer* = argmaxa Σr P(a, r | prompt)

Read it as pipeline stages:

StepWhat it isWhat it does
rA reasoning pathThe intermediate tokens. A latent variable — you don't care which one you got
aThe final answerThe bit you actually parse out and return
P(a, r | ·)Joint probabilityOne sample from the model is one draw from this
ΣrMarginaliseSum over paths. In practice: sample k of them and count
argmaxaTake the modeMajority vote over final answers. That's the whole algorithm

Plain chain-of-thought is the k=1 case with greedy decoding: one path, take its answer, hope it was a good one. Self-consistency is the same thing with the sum actually done.

The budget is a hard ceiling

The script implements an evaluator that may do exactly one operation per token it emits, and hands it a chain of six operations where each needs the previous one's result. This is an analogy for the depth limit of a fixed-size network, not a measurement of one. I'm labelling that hard because it's the one place on this page where you could mistake a toy for evidence.

Don't take my word for it. Here's the actual output:

token budget 1:  steps needed 6, steps available 1  ->  answer 7   (WRONG)
token budget 3:  steps needed 6, steps available 3  ->  answer 32  (WRONG)
token budget 6:  steps needed 6, steps available 6  ->  answer 63  (correct) ✓

the working, when it is allowed to exist:
   token 1: start 7 -> 7
   token 2: mul 3 -> 21
   token 3: add 11 -> 32
   token 4: sub 5 -> 27
   token 5: mul 2 -> 54
   token 6: add 9 -> 63
The evaluator at budget 1 isn't lazy and isn't ignorant. It ran out of serial depth. Every "just answer, don't explain" line you've put in a system prompt is you spending this budget, and you should know what it costs.

There is real theory for this now: Li et al. (2024) prove that constant-depth transformers with a chain of thought can solve serial problems that constant-depth transformers without one provably cannot. My evaluator is a cartoon of their result. Link at the bottom.

The prompts, side by side

Here are both prompts as the script builds them. Look for the difference. It is entirely in the exemplars' answers. 

--- STANDARD few-shot ---
Q: A shelf holds 4 boxes. Each box holds 6 mugs. 5 mugs break. How many
   are left?
A: The answer is 19.

--- CHAIN-OF-THOUGHT few-shot ---
Q: A shelf holds 4 boxes. Each box holds 6 mugs. 5 mugs break. How many
   are left?
A: 4 boxes * 6 mugs = 24 mugs. 24 - 5 broken = 19. The answer is 19.

prompt lengths: standard 301 chars, CoT 400 chars

Ninety-nine characters. That's the paper. (The script prints both full two-exemplar prompts; I've shown one exemplar each to fit the column.)

The claim you must not overstate. Wei et al.'s headline finding is an emergence result: chain-of-thought helps only at sufficient model scale, and on smaller models it can make performance worse than plain prompting. Nothing in my script demonstrates that — it needs real models at several sizes, which is what the paper has and I don't. Take the emergence claim from them and the mechanism from here.

Related, and frequently misattributed: the zero-shot "Let's think step by step" result is Kojima et al., 2022, not Wei — Wei's paper uses hand-written exemplars throughout. Worth having straight; it's an easy interview trip-up.

The bug that will actually bite you

Turning on chain-of-thought changes the shape of the model's output, and the answer parser you wrote for the old shape does not know that. This is the most common real failure in a CoT pipeline, and it does not throw: 

model output:
  9 pots * 8 cups = 72 cups. 72 - 14 spilled = 58. The answer is 58.

parse_loose  -> 9    <-- first number in the working
parse_strict -> 58 ✓  <-- anchored on 'the answer is'

Under the standard prompt the answer held exactly one number, so re.search(r"\d+") was correct and nobody reviewed it. Under CoT the same regex silently returns an intermediate value, accuracy drops, and the prompt gets blamed for a parser bug.

Self-consistency: why voting works

Wang et al., 2022 — arXiv 2203.11171.

This is the one piece of genuine statistics on the page, and the numbers mean what they say. The script's solver is stochastic: each of the six steps is carried out correctly 80% of the time, and a slip lands on a scattered wrong value. Sample k independent paths, take the modal answer.

The mechanism is one sentence: correct paths all agree on one number; wrong paths disagree with each other. So the mode beats any single draw, even when a single draw is right well under half the time. 

per-step accuracy 0.8, 6 steps, 4000 trials, true answer 63

  paths (k)   accuracy of the vote
          1   0.270
          3   0.332
          5   0.458
         11   0.745
         21   0.927
         41   0.994 ✓

one sample of 11 paths, so you can see the shape of it:
  answers: [77, 45, 81, 63, 51, 67, 69, 71, 66, 63, 87]
  vote   : 63  truth: 63

Note the gains flattening between 21 and 41. You're converging on the distribution's mode, not adding information, so the curve is bounded by "is the mode the right answer at all". Past that, more samples buy nothing but tokens.

And the failure mode people actually ship. Vote over paths that aren't independent and the vote does nothing: 

k=21, independent paths        : 0.928 ✓
k=21, 21 copies of ONE path    : 0.272   ← same as k=1

Same vote, same k, no benefit whatsoever. Sampling at temperature 0, or reusing one cached trace across your "samples", buys the bottom row and a 21× bill. Self-consistency needs independent errors — check that first when voting doesn't help.

ReAct: put a real tool in the scratch buffer

Chain-of-thought gives the model somewhere to write. ReAct's move is to let one of those writes be something the world said, not something the model produced. Thought, action, observation, repeat, until it answers or you cut it off.

Here's a full transcript from the script: two lookups against a real table and one call to a real calculator, both actually executed. The policy choosing the next action is a hand-written state machine — that's the part standing in for a model. The loop is the real thing. 

Question: How many more vans does Northwind have than Contoso?

  Thought 1: I need Northwind's van count. I should look it up, not recall it.
  Action  1: lookup('northwind', 'vans')
  Obs     1: 34
  Thought 2: Got Northwind. Now Contoso.
  Action  2: lookup('contoso', 'vans')
  Obs     2: 21
  Thought 3: Now subtract: 34 minus 21.
  Action  3: calc('34 - 21')
  Obs     3: 13
  Thought 4: The difference is 13.
  Answer: 13

finished in 4 loop iterations, answer = 13 ✓
ground truth = 13

Now remove one line — the one appending the observation to the scratchpad. The tool still gets called and the observation still gets printed. It just never reaches the thing deciding the next step: 

with observations fed back   : 13 ✓  ← checked
with observations discarded  : 15    ← guessed
ground truth                 : 13

  Action  1: lookup('northwind', 'vans')
  Obs     1: 34                          ← fetched, then thrown away
  Action  2: lookup('contoso', 'vans')
  Obs     2: 21                          ← fetched, then thrown away
  Thought 3: Now subtract: 40 minus 25.   ← plausible. memorised. wrong.
  Answer: 15

That fallback is a hardcoded number standing in for a memorised guess — I'm not claiming a model would produce 40. What the transcript shows is the shape of the failure: the fact was available, the call was made, and the answer still came from memory, because nothing wired the result back into the next step. Yao et al. report reduced hallucination for ReAct relative to chain-of-thought alone on the tasks they tested, and this is the plumbing reason why.

This is the same idea as RAG, moved inside the loop. RAG fetches once, up front, and stuffs the context. ReAct fetches repeatedly, deciding what to fetch from what it has already read. Same principle — put the fact in the context rather than trusting the weights — different control flow.

Put a limit on the loop

A ReAct loop with no step count is a while True with a credit card attached. Point it at a tool that keeps returning nothing and it never terminates on its own — the policy has no reason to stop asking: 

max_steps=None: still looping after 200 iterations, no answer, no end
                tool calls made: 200, distinct observations: 1
max_steps=5   : guard fired at step 5, returned None ✓

Two hundred identical tool calls, one distinct observation. The guard is three lines. Return None and let the caller decide what a non-answer means — don't let the loop decide for itself, and don't let it invent an answer to escape.

The code

NumPy and plain Python. No torch, no autograd, no model of any kind — the demos are the mechanism, printed. Every function short enough to read in one sitting.

Download the full script — 535 lines including comments, with eight runnable demos. Then: 

python3 chain_of_thought_from_scratch.py
def solve_with_token_budget(problem, budget):
    """Emit at most `budget` tokens; one op each; answer with what you have.

    Returns (answer, tokens_emitted, steps_needed, scratchpad).
    The scratchpad IS the emitted tokens -- it is the only memory there is.
    """
    steps_needed = len(problem)
    scratch = []
    state = None
    for op in problem[:budget]:
        state = apply_op(state, op)
        scratch.append(f"{op[0]} {op[1]} -> {state}")
    return state, min(budget, steps_needed), steps_needed, scratch





def react_loop(policy, question, max_steps=8, feed_observations=True,
               verbose=True):
    """Run thought/action/observation until the policy answers or we cut off.

    max_steps=None means no guard. Nothing stops it.
    """
    scratch = []          # the ONLY thing the policy gets to condition on
    steps = 0
    while max_steps is None or steps < max_steps:
        steps += 1
        thought, action = policy(question, scratch)
        if action[0] == "finish":
            return action[1], steps, scratch
        obs = run_action(action)

        # THE line that matters. With feed_observations=False the fact the
        # tool just fetched never reaches the scratchpad, so the next
        # thought is conditioned on a tool result the policy never saw.
        if feed_observations or action[0] == "calc":
            scratch.append((action, obs))
        else:
            scratch.append((action, None))
    return None, steps, scratch

Now break it

This is the part that actually makes it stick. Reading won't do it. You debug for a living — reverse-engineering from broken behaviour is your native mode, and every row below is something you can do to this file in under a minute.

Break thisWatch for
In demo_1, ask for the answer immediately: budget 1 instead of 6Answer 7 instead of 63. Six serial steps needed, one available. The budget ran out.
Swap parse_strict for parse_loose in your answer extractionReturns 9, not 58 — a number from inside the working. Silent. No exception. Your eval drops and the prompt gets blamed.
In demo_5, reuse one seed for all 21 pathsVote accuracy falls from 0.928 to 0.272 — exactly the k=1 number. Voting over correlated paths adds nothing.
Call react_loop(..., feed_observations=False)The tool is still called and still prints Obs 1: 34, but the loop answers 15 instead of 13. Fetched, then ignored.
Call react_loop(..., max_steps=None) with missing_key_policy200 identical tool calls, one distinct observation, no termination. With max_steps=5 the guard fires and returns None.
Set STEP_CORRECT_P = 0.35 and rerun demo_4The accuracy column stays near zero at every k. The mode of the samples is no longer the right answer, so more paths just converge on a wrong one, confidently.
Then read the papers, and not all of them. Wei et al.: the method section and the scale ablation — that's where the emergence claim lives, and it's the claim people get wrong. Skip the per-benchmark tables. Yao et al.: the method section and the hallucination analysis. Both are largely the comments you've already read, plus the experiments that make them credible.

On the maths anxiety

Good news: these two papers need almost none. No new loss, no new layer, no derivation. Chain-of-thought is a prompt format; ReAct is a while loop. If you can read the code above, you have read the methods.

The one bit that earns its keep is self-consistency, and it's first-year probability: a joint distribution over (answer, reasoning path), and marginalising — summing over the paths because you only care about the answer. "Marginalise" is a scary word for "group by the column you care about and sum the rest away". You've written that query.

What's genuinely hard here isn't maths at all, it's systems: token budgets, latency (k=41 paths is 41× the cost), tool timeouts, loop guards, answer parsing, and deciding when a non-answer beats a guess. That's engineering, and it's the half the papers don't cover.

Where to go next

This page gave you one angle: chain-of-thought as compute allocation, ReAct as the loop that makes the scratchpad factual. It skipped the experimental evidence entirely, because I can't produce that with NumPy. These four can.

Order I'd take them in: Wei → Yao → Weng → Li. The first three are readable in an evening and directly useful; save the complexity theory for when you want to know why it works rather than that it does.