Do you still need to tell an AI to think step by step?
For a few years, adding 'think step by step' to a prompt reliably improved answers on maths and logic tasks. Reasoning models now do something similar before they reply. That changes what the phrase is worth — and what you should write instead.
Where did 'think step by step' come from?
The phrase entered common use after researchers noticed that prompting a language model to reason through a problem before giving a final answer reduced errors on arithmetic and multi-step logic tasks. The model was forced to produce intermediate tokens — working, in effect — rather than jumping straight to an output. Those intermediate tokens then served as context for the final answer, making it more likely to be consistent.
The mechanism is straightforward: a model that writes 'first I need to find X, then I need to find Y' is less likely to contradict itself than one that produces an answer in a single pass. The phrase was a workaround for a real limitation — models trained to be concise would skip the working and get the answer wrong.
What changed with reasoning models?
Reasoning models are trained to produce an internal chain of thought before they surface a reply. That chain is not always shown to you, but it is there. The model is, in effect, already doing what 'think step by step' was designed to prompt. Repeating the instruction in your message is often redundant — like asking a calculator to 'please do the maths'.
Where it can actively get in the way is on tasks that do not require sequential reasoning. If you ask for a short summary and add 'think step by step', you may get a padded, over-structured response that buries the summary inside visible reasoning you did not need. The instruction changes the shape of the output, not just the quality of the thinking.
| Task type | Old completion model | Reasoning model |
|---|---|---|
| Multi-step maths | Phrase helps noticeably | Usually redundant |
| Formal logic puzzle | Phrase helps noticeably | Usually redundant |
| Short factual summary | Neutral or slight harm | Can bloat output |
| Creative writing | Usually unhelpful | Usually unhelpful |
| Ambiguous open question | Sometimes useful | Specify the goal instead |
What should you write instead?
The underlying goal of 'think step by step' was to get accurate, traceable answers. You can reach that goal more directly by telling the model what you actually want: the goal, any constraints, and the format you want back. If you need the working shown, say so explicitly — 'show your working before the final answer'. If you need a decision, say 'give me a recommendation and the two or three reasons behind it'. These are precise instructions rather than a general nudge.
Constraints matter as much as goals. Telling a model the word limit, the audience, or the decision it is feeding into shapes the output far more than a process instruction. A model that knows you are writing for a non-technical reader will calibrate differently than one given no context at all.
- Goal
- What the output is actually for — a decision, a draft, a check. State this first.
- Constraints
- Word count, audience, format, things to avoid. These narrow the space of valid answers.
- Format
- Whether you want bullet points, prose, a table, or working followed by a conclusion. Name it.
- Verification signal
- If accuracy matters, ask the model to flag where it is uncertain rather than paper over gaps.
Are there cases where the phrase is still worth using?
Yes. If you are working with a model you cannot identify — inside a tool, an API wrapper, or an embedded assistant — you do not know whether it reasons internally. In that case, 'think step by step' is a reasonable fallback. It costs little and may help. The same applies when you are iterating quickly and do not want to write a full structured prompt for a one-off question.
It also remains useful as a diagnostic. If a model is giving you inconsistent answers, adding the phrase and looking at the intermediate steps can reveal where the reasoning breaks down. That is not about improving the answer — it is about understanding the failure. For that purpose, the phrase is as useful as it ever was.
GROX routes each request to the appropriate model and capability behind a single conversational surface, so you rarely need to think about which underlying model is handling a given task. When you do need to inspect reasoning or choose an engine explicitly — as in GROX Code — the interface surfaces that choice rather than hiding it.
Common questions
Does 'think step by step' still improve accuracy?
On older completion-style models, yes, reliably. On modern reasoning models, the effect is much smaller because the model already generates an internal chain of thought before replying. For most tasks, stating your goal and the format you want back will do more than adding a process instruction.
Can the phrase make answers worse?
It can. On tasks that do not require sequential reasoning — summaries, short factual answers, creative drafts — the instruction can produce padded, over-structured output. The model may surface visible reasoning you did not need, burying the actual answer. If you want brevity, ask for brevity explicitly rather than adding a step-by-step instruction.
What is the simplest replacement for 'think step by step'?
State what the output is for, any hard constraints, and the format you want back. If you need the working shown, say 'show your working before the final answer'. If you need a recommendation, say so. Specific instructions outperform general process nudges on most tasks with reasoning models.
Should I use the phrase when I do not know which model is running?
It is a reasonable fallback. If you cannot tell whether the underlying model reasons internally, adding 'think step by step' costs little and may improve accuracy on logic or maths tasks. Treat it as a safe default for unknown environments rather than a necessary step for all prompts.
If you want to explore how a persistent agent handles prompting, memory and task routing in practice, GROX has a free tier with no card required.