Few-Shot Prompting

What examples do to a model

Few-shot prompting means placing a few input/output pairs in the prompt so the model copies the pattern on the real input. Nothing about the model changes; OpenAI's guide describes it as the model implicitly picking up the pattern from the examples and applying it to the new prompt. Examples are the strongest lever for output format, label vocabulary, and tone. When instructions alone cannot pin down a format, two or three examples usually can.

Mirror your real inputs

Examples teach the model what your actual data looks like. Anthropic's best practices list relevance first: examples should mirror the real use case closely. A classifier trained on examples with short clean sentences will stumble on the messy, typo-filled tickets your users actually write. Pull examples from production logs, anonymize them, and use those. Synthetic toy examples teach the model a task that does not exist.

Cover the edges, not just the easy cases

Diversity is the second quality Anthropic asks for: vary the examples enough that the model does not latch onto surface patterns. Include the ambiguous case, the empty input, the near-duplicate that deserves a different label. Balanced classes matter too; five positive examples and one negative example teach the model that everything is positive.

```python

from openai import OpenAI

client = OpenAI()

messages = [

{"role": "system", "content": "Label the review Positive, Negative, or Neutral. Reply with one word."},

{"role": "user", "content": "Setup took five minutes and it just works."},

{"role": "assistant", "content": "Positive"},

{"role": "user", "content": "It works, but the mobile app crashes daily."},

{"role": "assistant", "content": "Neutral"},

{"role": "user", "content": "Charged me after I cancelled. Avoid."},

{"role": "assistant", "content": "Negative"},

{"role": "user", "content": "The export feature is slow on large files."},

]

response = client.chat.completions.create(

model="gpt-4o-mini",

temperature=0,

max_tokens=10,

messages=messages,

)

print(response.choices[0].message.content)

```

Notice the mixed review labeled Neutral. That one example does more work than two more obvious positives would.

Format examples so the pattern is visible

Keep every example in the same shape: same labels, same order, same delimiters. In chat APIs, user/assistant message pairs are the natural container, as above. In a single text prompt, wrap each example in XML tags so the model can tell examples apart from instructions. Anthropic recommends <example> tags nested inside <examples>:

```

<examples>

<example>

<input>Release 2.4 adds dark mode and fixes the export bug.</input>

<output>- Added dark mode

• Fixed the export bug</output>

</example>

<example>

<input>Release 2.5 speeds up search and removes the legacy API.</input>

<output>- Sped up search

• Removed the legacy API</output>

</example>

</examples>

```

Consistency is what the model copies, including your mistakes. One example with a trailing period and one without teaches the model that punctuation is optional.

Use examples to set tone

Examples steer tone and structure, not just labels. Anthropic's docs list tone alongside format as things examples control. Two sentences in the voice you want beat a paragraph describing it:

```

<example>

<input>Write a release note for the new export feature.</input>

<output>Export is 4x faster on files over 100MB. Update to 2.5 to get it.</output>

</example>

```

The model copies the plain register, the short sentences, the absence of marketing adjectives. When the output keeps sounding wrong despite good instructions, add tone examples before rewriting the instructions.

Refresh examples as the task changes

Examples rot. The ticket categories you launched with are not the categories you have after a year, and a stale example set quietly teaches yesterday's labels. Re-sample examples from recent production traffic on a schedule, and drop any example that no longer represents what the model should do. An evaluation set built from the same old inputs will not catch this; the examples and the evals both need fresh data.

Keep the count small

Three to eight examples cover most tasks. Each example is tokens you pay for on every request, and returns shrink past the point where the pattern is clear. If the task needs more than eight examples to behave, the pattern is probably too subtle for examples alone, and the task wants either clearer instructions or fine-tuning.

Watch what the examples accidentally teach

Models copy everything in the examples, not just the intended pattern. Check for three failure modes: labels that disagree with each other, examples whose style varies (one terse, one chatty), and examples where the input contains a word the model then repeats in unrelated answers. When output goes wrong, read your examples with fresh eyes before blaming the model; most few-shot bugs are in the examples.

Few-shot or fine-tuning

Use few-shot prompting while the task is still changing: new labels, new formats, low volume. The examples live in the prompt, so iterating costs nothing but an edit. Move to fine-tuning when the pattern is stable, the call volume is high enough that per-request example tokens dominate your bill, or the examples would need to exceed what fits comfortably in context. Few-shot is the prototype; fine-tuning is the production lock-in. Prototype first.