Few-Shot Learning

Few-shot learning is a model's ability to perform a task after seeing only a few worked examples. With large language models it usually means few-shot prompting: you place a handful of input and output pairs in the prompt, then give a new input, and the model continues the pattern without any change to its weights.

The idea predates LLMs. In classic machine learning, few-shot learning referred to training classifiers that generalize from a few labeled examples per class, often through meta-learning. The LLM sense took off with OpenAI's 2020 GPT-3 paper, Language Models are Few-Shot Learners. The authors tested the 175-billion-parameter model in three settings: zero-shot (an instruction only), one-shot (one demonstration) and few-shot, where they included as many demonstrations as would fit in the model's 2,048-token context window, typically 10 to 100. They called this in-context learning, and showed that few-shot performance sometimes approached fine-tuned models.

Modern models need far fewer examples. Instruction tuning and RLHF mean current models handle many tasks well zero-shot, and context windows of around a million tokens remove the old space limit. Few-shot prompting is still the most reliable way to pin down things an instruction describes poorly: an exact output format, a house writing style, the boundary between two labels, or how to handle an edge case. Two to five well-chosen examples usually do more than a long paragraph of rules.

A few practical rules hold across models. Examples should cover the variation you expect, including at least one hard case, because models copy surface features: if every example is short, outputs will be short. Order matters, and the last example tends to carry the most weight. Labels in examples must be correct, since the model learns from mistakes as readily as from good examples. For tasks requiring multi-step reasoning, examples that show the reasoning (chain-of-thought examples) help more than examples that show only the answer. And when a task needs hundreds of examples to get right, fine-tuning or a dedicated classifier is usually cheaper than very long prompts.

Few-shot prompting shows up throughout GEO work. Teams use it to classify AI answers at scale, for example deciding whether a response recommends a brand, merely lists it, or warns against it, where a short instruction leaves too much room for interpretation. It is used to generate realistic prompt sets for tracking, by showing the model a few real customer questions and asking for more in the same voice. And it keeps AI-assisted content consistent with a style guide across hundreds of pages. In each case, the examples are effectively a specification: when results drift, fix the examples before rewriting the instructions.

The same mechanism explains a less obvious point about how models read web content. A page that presents information in a consistent, repeated structure, such as a set of question and answer pairs or a specification table, gives a model a clear pattern to extract from, much as a few-shot prompt does.