Why prompts matter: what the research says about getting better AI answers

It's tempting to think of AI tools like ChatGPT and Claude as magic boxes: type anything, get a smart answer. But anyone who has used them for a while notices something odd. Ask the same thing two different ways and you can get two very different answers, one vague and generic, the other genuinely useful.

That isn't just a feeling. Researchers have measured it, and the results are striking. This guide walks through four studies that show why the way you write your request, your prompt, matters so much, and what that means for you as an everyday user.

Study 1: Five words that quadrupled the accuracy

In 2022, a team of researchers from the University of Tokyo and Google published a paper called "Large Language Models are Zero-Shot Reasoners", presented at NeurIPS, one of the most respected AI conferences. They tested a simple idea: what happens if you add the phrase "Let's think step by step" before the AI answers a math word problem?

The result surprised a lot of people. On one set of arithmetic word problems (called MultiArith), the AI's accuracy jumped from 17.7% to 78.7%. On a harder set of grade-school math problems (GSM8K), it rose from 10.4% to 40.7%. Same AI model, same questions. The only change was a short instruction asking it to work through the problem before answering. You can read the paper on arXiv.

The lesson for beginners isn't that "Let's think step by step" is a magic spell. It's that the AI does a better job when you tell it how to approach the task, not just what the task is.

One honest caveat: newer AI models often reason step by step on their own, so this exact phrase matters less than it did in 2022. But asking for the working still helps you, because you can spot where an answer went wrong.

Study 2: Tiny formatting changes, huge differences

If wording matters, does formatting matter too? A 2023 study by researchers at the University of Washington and the Allen Institute for AI, later presented at ICLR 2024, tested exactly that. They took prompts and changed only things that shouldn't affect the meaning, like spacing, punctuation, or how labels were written.

They found that some widely used AI models were "extremely sensitive" to these small changes, with performance differences of up to 76 accuracy points on one model. The paper's subtitle sums it up: "How I learned to start worrying about prompt formatting." You can read it on arXiv.

For you, the takeaway is simple: clear structure helps. When a request has several parts, putting them on separate lines or in a short list makes it easier for the AI to follow every part.

Study 3: Harvard and BCG put 758 consultants to the test

The most famous real-world study comes from Harvard Business School and Boston Consulting Group. In 2023, researchers ran a randomized experiment with 758 BCG consultants, about 7% of the firm's individual contributor consultants. Participants were split into three groups: no AI, GPT-4 access, or GPT-4 access plus a short prompt engineering overview.

On 18 realistic consulting tasks that fell within what AI does well, consultants using AI completed 12.2% more tasks, finished them 25.1% faster, and produced work that graders rated more than 40% higher in quality than the group without AI. You can read the summary on the Harvard Business School site.

But there was a twist. On a task deliberately chosen to fall outside what AI handled well, consultants using AI were 19% less likely to reach the correct answer. The researchers called this the "jagged technological frontier": AI is excellent at some tasks and quietly wrong at others that look just as hard.

That's why good prompting isn't only about getting a polished answer. It's also about knowing when to double-check, asking the AI to flag uncertainty, and giving it the facts it needs rather than letting it fill gaps on its own. Our guide on how to stop ChatGPT making things up covers this in detail.

Study 4: There's no magic formula, and that's good news

You'll see plenty of posts online promising "secret" prompt tricks. A 2025 report from researchers at the Wharton School, "Prompt Engineering is Complicated and Contingent", tested some popular ideas, such as being polite to the AI or tightly constraining its answers. They found that these tricks sometimes helped and sometimes hurt, depending on the question. Read it on arXiv.

This is actually reassuring for beginners. You don't need to memorize hacks. What consistently works is the boring stuff: being clear about what you want, giving useful context, and saying what a good answer looks like.

What all of this means for you

Put the four studies together and a clear picture emerges:

1. Your words change the result. The same AI can give a weak answer or a strong one depending on how you ask.
2. Telling the AI how to work helps. Asking it to reason step by step, list assumptions, or check its own work improves answers, especially for anything involving numbers or logic.
3. Structure beats rambling. Short, organized prompts are easier for AI to follow than one long run-on sentence.
4. AI can be confidently wrong. Good prompts ask it to flag uncertainty, and good users verify anything important.
5. Clarity beats tricks. There are no secret passwords, just clear instructions and good context.

The good news is that these are learnable habits, not technical skills. If you can explain a task clearly to a new coworker, you can write a strong prompt.

Where to go next

If you're brand new, start with what a prompt is, then learn the 5-part prompt formula that turns any vague request into a clear one. When you're ready, read about the most common prompt mistakes so you can avoid them from day one.

Frequently asked questions