in-context learning (ICL)

In-context learning (ICL) is when a pretrained model performs a new task by conditioning on natural-language instructions and (optionally) input-output demonstrations included in the prompt, without updating its internal parameters. Demonstrations were traditionally limited to a handful, but larger context windows have made many-shot ICL practical. A single prompt can now carry hundreds or even thousands of examples.

ICL enables what appear as zero-shot or few-shot capabilities by treating the prompt itself as a temporary training set.

In practice, the success of ICL depends heavily on different factors, such as how many examples are included, which ones are chosen and how diverse they are, their order, how the instruction is phrased, and how the prompt is formatted.