It’s Not What You Ask, It’s How You Ask: The Science of Paraphrasing Prompts

Introduction

“It’s not what you say, it’s how you say it.”

This age-old adage usually applies to human relationships, implying that tone and delivery matter as much as the message itself. Surprisingly, this rule applies just as strictly to Large Language Models (LLMs).

If you have ever spent hours tweaking a prompt for ChatGPT or LLaMA—changing a word here, adding a “please” there—you have engaged in the often frustrating art of prompt engineering. We know that slight variations in instructions can lead to wildly different outputs, but until recently, this process has been largely based on intuition and trial-and-error.

What if we could turn that art into a science? What if we knew exactly which linguistic changes—like swapping a noun, changing a verb tense, or altering sentence structure—actually help a model perform better?

A recent research paper titled “Paraphrase Types Elicit Prompt Engineering Capabilities” by Jan Philip Wahle and colleagues tackles this exact problem. By systematically testing hundreds of thousands of prompt variations across 120 tasks, the researchers have provided a roadmap for understanding how linguistic nuances shape machine intelligence.

In this deep dive, we will explore their methodology, dissect their findings, and reveal why “paraphrasing” might be the most powerful tool in your prompt engineering toolkit.

The Core Problem: The Sensitivity of LLMs

LLMs mimic human interaction by following instructions. However, unlike humans, who can usually infer intent regardless of phrasing, LLMs can be incredibly sensitive to syntax and vocabulary.

Consider a simple directive:

1. “Avoid procrastination.”
2. “Stop postponing what you have to do.”

To a human, these sentences are semantically identical. To an LLM, they are two distinct vector paths that could lead to different quality outputs. The researchers posit that paraphrasing provides a “window into the heart of prompt engineering.” By analyzing how models react to specific types of paraphrases, we can identify which linguistic features the models value, what they understand, and where they fail.

Methodology: Systematizing the Prompt

The researchers did not just randomly rewrite prompts; they applied a rigorous taxonomy of linguistic changes. They evaluated five major models (Command R+, LLaMA 3 70B, LLaMA 3 8B, Mixtral 8x7B, and Gemma 7B) across 120 different NLP tasks (ranging from sentiment analysis to code generation).

The Paraphrase Pipeline

The study’s core method involved taking an original task prompt and generating 26 specific variations of it using a controlled generation model. These variations were categorized into six “families” of linguistic changes.

As shown in Figure 2, the process flows from the original prompt through specific linguistic filters. Here is a breakdown of the categories they tested:

1. Morphology: Changes to the form of words (e.g., changing “assign” to “assignment”, or “must” to “should”).
2. Lexicon: Vocabulary changes. Replacing words with synonyms or changing the specificity (e.g., changing “text” to “paragraph”).
3. Syntax: Grammatical structural changes (e.g., changing active voice to passive voice, or altering negation).
4. Lexico-Syntax: A hybrid of word choice and structure (e.g., swapping “buy” for “purchase” which might alter the surrounding preposition usage).
5. Discourse: Changes to the flow or style, such as punctuation or changing from direct to indirect speech.
6. Others: Reordering words or adding/deleting non-essential information.

To visualize what a specific prompt looks like in this study, consider the template used for the “NumerSense” task in Figure 10 below. The model must predict a missing number. The researchers would take the instruction text in this template and apply the linguistic changes described above.

Results: How Much Does Phrasing Matter?

The headline result is clear: Paraphrasing prompts can lead to massive performance gains.

The researchers measured “Potential Gain,” which is the improvement in performance (using the ROUGE-L metric) that could be achieved if the optimal paraphrase type was selected for a specific task.

As Figure 1 illustrates, the potential upside is significant:

• Gemma 7B showed a massive 13.4% potential gain.
• Mixtral 8x7B showed a 6.7% gain.
• LLaMA 3 8B showed a 5.5% gain.

This graph suggests that current model performance is often a “lower bound.” The intelligence to solve the task is present in the model, but it requires the specific “key”—the right linguistic phrasing—to unlock it.

Which Linguistic Levers Pull the Most Weight?

Not all paraphrases are created equal. Some changes confuse the model, while others clarify the instruction. The study decomposed the performance impact by linguistic group.

Figure 3 reveals the hierarchy of impact:

1. Lexicon Changes (+1.26% median gain): Simply swapping vocabulary words (e.g., “detect” vs. “find”) has the highest median impact. This suggests that models have “favorite words” or specific associations with certain terms that trigger better capabilities.
2. Syntax Changes (+1.19% median gain): Altering sentence structure helps significantly. This could be as simple as moving clauses around to prioritize the most important instruction.
3. Morphology (-0.87% median loss): interestingly, morphological changes (like changing verb tenses or singular/plural forms) often hurt performance on average, though they can be highly effective in specific niches.

To understand this granularity better, we can look at the specific types of changes within these groups.

Figure 11 details these sub-types. Notice how “Same Polarity Substitution” (swapping a positive word for another positive synonym) yields a +1.32% gain, while “Inflectional changes” (like changing run to running) result in a loss. This nuance is critical: it implies that maintaining the sentiment and intent (polarity) while refreshing the vocabulary is a winning strategy.

The Scaling Law of Prompts

Does a smarter model need less prompt engineering? The data suggests yes.

The study found a strong correlation between model size and sensitivity to paraphrasing. Smaller models (like Gemma 7B) are volatile; they can fail hard with bad prompts but soar with good ones. Larger models (like LLaMA 3 70B) are more robust—they are less likely to be confused by poor phrasing, but they also gain less from perfect phrasing.

Figure 12 visualizes this comparison. The blue dots (LLaMA 3 8B) are scattered much wider than the orange dots (70B).

• Takeaway: If you are deploying a smaller, cheaper model to save on inference costs, your prompt engineering strategy becomes mission-critical. You can essentially “punch up” a weight class by optimizing your prompts, bringing an 8B model’s performance closer to a 70B model’s baseline.

Why Does Paraphrasing Work? (Debunking Hypotheses)

The researchers meticulously checked for confounding factors to explain why these changes improved performance.

Hypothesis 1: It’s just memorization.

Theory: Maybe the paraphrased prompts just happen to look more like the data the model was trained on (e.g., CommonCrawl or Wikipedia), so the model completes them more easily. Finding: No.

The researchers compared the prompts against the FineWeb corpus (350 billion tokens). They calculated \(\Delta_{train}\)—the difference in similarity to training data between the original and paraphrased prompt.

Figure 5 shows the result. If the “memorization” theory were true, we would see a cluster of high-performance points (red/yellow) on the far right side of the x-axis (high similarity). Instead, high performance is distributed vertically. The most successful prompts were often not the ones closest to the training data.

Hypothesis 2: Complexity and Length.

Theory: Maybe longer, more verbose prompts give the model “more time to think,” or perhaps shorter, punchier prompts are clearer. Finding: No correlation.

Table 2 shows Pearson correlations between prompt complexity metrics (like token count) and performance. The values are consistently near zero (e.g., 0.02). This indicates that simply adding more words or making the prompt “complex” does not inherently lead to better results. It is the semantics of the change, not the length, that matters.

Hypothesis 3: It’s just randomness (Temperature).

Theory: LLMs are probabilistic. Maybe the “better” prompt just got lucky with the random seed? Finding: Mostly false.

Figure 7 plots performance differences across temperature settings (from 0.0 to 1.0). While high temperatures introduce randomness, the study found that performance gains from paraphrasing persisted even at low temperatures (where the model is most deterministic). This confirms that the linguistic signal in the prompt is driving the improvement, not rolling the dice.

The Cost of Creativity: Lexical Diversity

There is one final trade-off to consider. If we optimize a prompt for accuracy (getting the “right” answer), do we lose creativity?

The researchers measured Lexical Diversity using metrics like RTTR (Root Type-Token Ratio) to see if the model’s output became repetitive or boring when accuracy went up.

Figure 6 reveals a mixed bag. For tasks like Summarization (magenta dots), you can hit the “sweet spot” (top right quadrant) where you get both better performance and high lexical diversity. However, for Question Rewriting (red dots), gains in accuracy often resulted in lower lexical diversity.

This implies that if your goal is creative writing, you might need different paraphrase strategies than if your goal is exact logic or data extraction.

Conclusion

The paper “Paraphrase Types Elicit Prompt Engineering Capabilities” moves us away from the “black magic” view of prompting toward a more engineered approach.

Here are the key takeaways for students and practitioners:

1. Don’t Settle for the First Draft: Your first prompt is rarely the best. Paraphrasing is a valid, high-impact optimization strategy.
2. Focus on Lexicon and Syntax: Don’t just add instructions; try swapping verbs and changing sentence structures. These yields the highest returns.
3. Small Models Need Better Prompts: If you are working with limited compute (e.g., LLaMA 8B or Gemma 7B), prompt engineering is your most effective lever for performance improvement.
4. It’s Not About Length: Making a prompt longer doesn’t make it smarter. Making it linguistically precise does.

As LLMs continue to evolve, the interface between human intent and machine execution remains natural language. Understanding the “linguistics of prompting” is no longer just for NLP researchers—it is a requisite skill for anyone building on top of these powerful models.