A Strange Instruction That Actually Improves Answers
If you’ve spent any time using AI chatbots, you may have heard the advice to add “think step by step” or “explain your reasoning before giving the answer” to your prompt. It sounds like a superstition, the kind of trick people repeat without understanding why it works. But chain-of-thought prompting is a real, measurable technique, and understanding why it helps will change how you write prompts for everything from math problems to business decisions.
What’s Actually Happening Under the Hood
Large language models generate their response one word (technically, one token) at a time, and each new token is chosen based on everything written so far, including the model’s own prior output. When you ask a model to jump straight to a final answer on a multi-step problem, it has to somehow pack all the necessary reasoning into that single moment of generating the first token of the answer, with no scratch space to work through intermediate steps. When you instead ask it to reason aloud first, each step of reasoning becomes additional context the model can reference while generating the next step, functioning like a visible scratchpad. The model isn’t smarter when you ask it to show its work; it’s doing less reasoning “in its head” and more reasoning “on paper,” which plays to how these models actually process information.
Where It Helps Most (and Where It Doesn’t)
Chain-of-thought prompting shows the biggest gains on problems with multiple steps: word math problems, logic puzzles, multi-criteria comparisons, debugging code, or any task where the final answer depends on correctly combining several pieces of information. For simple factual lookups or single-step questions, it adds little value and just makes the response longer. A good rule of thumb: if you’d need a pen and paper to solve the problem yourself, ask the AI to reason step by step too. If you could answer it instantly without working anything out, skip the extra instruction.
How to Use It in Practice
You rarely need the exact phrase “think step by step” anymore, since most modern assistants reason internally by default on complex requests. But you can still steer this explicitly when it matters: ask the model to “list your assumptions before answering,” “walk through the trade-offs of each option,” or “show the calculation, then give the final number.” This is especially useful when you plan to verify the answer yourself, since a visible chain of reasoning lets you catch a wrong assumption partway through, rather than only discovering an error in the final answer with no way to tell where it went wrong.
