REF-026: A Survey on In-Context Learning

REF-026: A Survey on In-Context Learning

Citation

Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., Sun, X., Li, L., & Sui, Z. (2024). A Survey on In-Context Learning. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), 1107-1128.

Paper: https://aclanthology.org/2024.emnlp-main.64/

arXiv: https://arxiv.org/abs/2301.00234

Document Profile

AttributeValue
Pages22 (1107-1128 in proceedings)
Year2024 (arXiv: 2022, published 2024)
VenueEMNLP 2024 (Conference on Empirical Methods in Natural Language Processing)
TypeComprehensive survey paper
AIWG RelevanceCritical - Foundational for template design, prompt engineering, few-shot patterns

Executive Summary

This comprehensive survey examines in-context learning (ICL) - the ability of large language models to learn tasks from a few examples provided in the prompt without parameter updates. The paper presents the first systematic survey of ICL research, covering formal definitions, training strategies, prompt design techniques, theoretical analysis, and applications across modalities.

Core Finding: ICL enables LLMs to perform complex tasks by learning from analogy through demonstration examples. The effectiveness of ICL depends critically on: (1) demonstration selection and ordering, (2) instruction formatting, (3) model pretraining characteristics, and (4) understanding that format matters more than label correctness.

AIWG Significance: ICL research directly informs AIWG's template design philosophy. The finding that "demonstration structure is more important than label correctness" validates AIWG's emphasis on consistent template formats with well-structured examples. The survey's taxonomy of demonstration selection, reformatting, and ordering provides empirical grounding for AIWG's few-shot template patterns.

Key Concepts

1. In-Context Learning (ICL)

Definition: "A paradigm that allows language models to learn tasks given only a few examples in the form of demonstration." (p. 1108)

ICL differs from traditional learning:

  • No parameter updates - Model weights remain frozen
  • Learning from analogy - Similar to human reasoning by example
  • Training-free - Adapts to new tasks without gradient descent

Formal Definition:

P(y_j | x) = f_M(y_j, C, x)

where:
- C = demonstration set {I, s(x1,y1), ..., s(xk,yk)}
- I = optional task instruction
- s(xi,yi) = example formatted in natural language
- f_M = scoring function

2. Demonstration Components

Prompt Structure (Figure 1, p. 1107):

[Instruction] (optional)
"Classify the sentiment of the text"

[Demonstrations] (k examples)
Review: "Delicious food!" Sentiment: Positive
Review: "The food is awful." Sentiment: Negative
...

[Query]
Review: "Good meal!" Sentiment: ?

3. ICL Taxonomy (Figure 2, p. 1108)

The survey organizes ICL research into three stages:

Training Stage:

  • Pre-training: Reorganizing corpora for ICL (PICL, ICLM)
  • Warmup: Instruction tuning, meta-learning (MetaICL, FLAN)

Inference Stage:

  • Demonstration Organization: Selection, reformatting, ordering
  • Instruction Formatting: Task descriptions, chain-of-thought
  • Scoring Functions: Direct, PPL, Channel models

Analysis:

  • Influencing Factors: Pre-training data, model architecture, demonstration properties
  • Learning Mechanisms: Induction heads, Bayesian inference, gradient descent analogy

4. Demonstration Selection Methods

Unsupervised Methods:

  • KATE (k-Nearest Neighbors): Select demonstrations similar to query
  • Mutual Information: Choose examples maximizing MI with query
  • Perplexity-based: Use LLM perplexity scores for selection
  • Votek: Graph-based diverse and representative selection

Supervised Methods:

  • EPR: Train dense retriever with task-specific supervision
  • Q-Learning: Frame selection as Markov decision process
  • AdaICL: Model-adaptive selection using uncertainty scores

5. Scoring Functions (Table 3, p. 1111)

MethodTargetEfficiencyCoverageStability
DirectM(y_j \C, x)+++ (fastest)+ (local tokens)+ (sensitive)
PPLPPL(S_j)+ (slower)+++ (all tokens)+ (sensitive)
ChannelM(x \C, y_j)+ (slower)+ (local tokens)++ (more stable)

Key Trade-offs:

  • Direct: Fast but restricts template design (answer must be at end)
  • PPL: Universal but computationally expensive
  • Channel: Robust under imbalanced data but requires generating full input

6. Emergent Abilities

Scale-dependent ICL (Wei et al., 2022b):

  • ICL capability emerges at certain model scale thresholds
  • Both model parameters and pretraining steps matter
  • Demonstrates "emergent abilities" of large language models

7. Chain-of-Thought (CoT) in ICL

CoT Prompting (Wei et al., 2022c):

  • Adds intermediate reasoning steps between input and output
  • Significantly improves performance on complex reasoning tasks
  • Example: Math word problems, commonsense reasoning

Auto-CoT Methods:

  • LLMs can generate reasoning paths automatically
  • Self-consistency improves CoT reliability
  • Least-to-most prompting for compositional tasks

Key Findings

1. Format Matters More Than Label Correctness

Finding: "The label space and the distribution of the input text specified by the demonstrations are both important—regardless of whether the labels are correct for individual inputs." (Min et al., 2022c, cited p. 1113)

Evidence:

  • Experiments show ICL works even with random labels if format is consistent
  • Input distribution and label space matter more than individual correctness
  • Later studies qualified this: accurate mappings DO help, but format is critical baseline

Implications for AIWG: Template structure consistency is paramount; examples serve to establish format patterns.

2. Demonstration Order Sensitivity

Finding: "Order sensitivity is a common problem and always exists for various models." (Lu et al., 2022, p. 1111)

Evidence:

  • Different orderings of the same demonstrations produce significantly different results
  • Global and local entropy metrics correlate with ICL performance
  • Simple-to-complex ordering (ICCL) improves performance

Design Strategies:

  • GlobalE & LocalE: Use entropy metrics to determine optimal ordering
  • ICCL: Arrange from simple to complex examples
  • Proximity-based: Closest example as rightmost demonstration

3. Model Warmup Enhances ICL

Finding: "All these warmup methods improve the ICL capability by updating the model parameters, which implies that the ICL capability of the original LLMs has great potential for improvement." (p. 1124)

Warmup Methods:

  • Instruction Tuning (FLAN): 60+ datasets with natural language instructions
  • MetaICL: Multi-task learning with demonstration formats
  • Symbol Tuning: Replace natural labels with arbitrary symbols (e.g., "foo/bar")

Performance Gains:

  • FLAN improves zero-shot and few-shot on unseen tasks
  • Scaling to 1000+ instructions further improves generalization
  • Warmup plateau: Small amounts of data sufficient for adaptation

4. Pretraining Data Distribution Critical

Finding: "ICL capability emerges when the training data exhibits specific distributional properties, such as burstiness, wherein items appear in clusters rather than being uniformly distributed." (Chan et al., 2022, p. 1112)

Contributing Factors:

  • Diversity: Multiple corpora more important than size alone
  • Burstiness: Clustered examples better than uniform distribution
  • Domain: Source domain matters more than corpus size
  • Task Diversity Threshold: Beyond threshold, strong generalization emerges

5. Induction Heads as Functional Mechanism

Finding: "Specific attention heads, referred to as 'induction heads', can replicate previous patterns for next-token prediction, thus progressively developing ICL capabilities." (Olsson et al., 2022, p. 1113)

Mechanism:

  • Induction heads identify and copy patterns from demonstrations
  • Computational layers show information flow: demonstrations → labels → prediction
  • Label words serve as "anchors" aggregating information

Theoretical Importance: Provides mechanistic explanation for how ICL works at the architecture level.

6. ICL as Implicit Bayesian Inference

Finding: "ICL is explained as implicit Bayesian inference, where models perform ICL by identifying a shared latent concept among examples." (Xie et al., 2022; Jiang, 2023, p. 1113)

Bayesian Framework:

P(y | x, D) = ∫ P(y | x, θ) P(θ | D) dθ

where:
- D = demonstration set
- θ = latent concept/task
- P(θ | D) = posterior over concepts given demonstrations

Interpretation: LLMs infer task concept from demonstrations and apply to query.

7. ICL Resembles Gradient Descent

Finding: "GPT-based ICL behaves similarly to explicit finetuning from multiple perspectives." (Dai et al., 2023a, p. 1113)

Dual Form:

  • Transformer attention has dual form with gradient descent
  • One step of gradient descent is provably optimal for linear self-attention
  • ICL can be viewed as "meta-optimizers" performing implicit gradient descent

Controversy: Simplified settings (linear regression) may not fully generalize to real-world scenarios.

8. Demonstration Selection: Model-Dependent Effectiveness

Finding: Experimental comparison (Table 2, p. 1111) shows demonstration selection methods are model-dependent.

Results:

Modeltopkvotekmdl
GPT-249.534.954.4
GPT-J63.950.663.5
Qwen269.467.069.2
Llama371.570.570.9

Conclusion: No single selection method dominates; effectiveness depends on model architecture and scale.

Benchmark and Experimental Results

Demonstration Selection Comparison (Table 2, p. 1111)

Datasets: SST-5, SST-2, CommonsenseQA, SNLI, AG News (8 demonstrations)

Average Performance:

  • GPT-2: topk (49.5) < mdl (54.4)
  • GPT-J: topk (63.9) ≈ mdl (63.5) > votek (50.6)
  • Qwen2: topk (69.4) ≈ mdl (69.2) > votek (67.0)
  • Llama3: topk (71.5) > mdl (70.9) > votek (70.5)

Takeaway: Simple topk (k-nearest neighbors) performs competitively or best for larger models; mdl excels for smaller models.

Scoring Function Efficiency (Table 4, p. 1126)

Inference Latency (ms) with 8 in-context examples:

ModelDirectPPLChannel
GPT-244.13 (1.00×)114.02 (2.58×)157.70 (3.57×)
GPT-J611.04 (1.00×)1766.82 (2.89×)1793.27 (2.93×)
Qwen2745.89 (1.00×)1886.63 (2.53×)1957.97 (2.63×)
Llama3790.46 (1.00×)1935.04 (2.45×)1956.21 (2.47×)

Takeaway: Direct scoring is 2.5-3× faster than PPL or Channel; efficiency trade-off for flexibility.

Scoring Function Stability (Table 5, p. 1126)

Variance across 5 random seeds (lower = more stable):

ModelDirectPPLChannel
GPT-21.120.853.18
GPT-J1.000.774.06
Qwen20.720.702.43
Llama30.890.782.43
Average0.930.783.03

Takeaway: PPL is most stable (0.78 avg variance); Channel is least stable (3.03), suggesting sensitivity to demonstration changes.

New Challenging Benchmarks (Table 6, p. 1126)

BenchmarkTasksFocus
BIG-Bench204Mixed tasks across domains
BBH (BIG-Bench Hard)23Unsolved problems where LLMs fail
PRONTOQA1Logical reasoning and QA
MGSM1Multilingual math reasoning
LLMAS8Planning and reasoning about change
OPT-IML Bench2000Cross-task generalization

Progress: Best models now outperform average human raters on 65% of BIG-Bench tasks via ICL.

Key Quotes for Citation

"In-context learning is a paradigm that allows language models to learn tasks given only a few examples in the form of demonstration." (p. 1108)

"The key idea of in-context learning is to learn from analogy." (p. 1107)

"Different from supervised learning, which requires a training stage that uses backward gradients to update model parameters, ICL does not perform parameter updates. The model is expected to learn the pattern hidden in the demonstration and accordingly make the right prediction." (p. 1107)

"The label space and the distribution of the input text specified by the demonstrations are both important—regardless of whether the labels are correct for individual inputs." (Min et al., 2022c, as cited on p. 1113)

"Order sensitivity is a common problem and always exists for various models." (Lu et al., 2022, p. 1111)

"Specific attention heads, referred to as 'induction heads', can replicate previous patterns for next-token prediction, thus progressively developing ICL capabilities." (Olsson et al., 2022, p. 1113)

"GPT-based ICL behaves similarly to explicit finetuning from multiple perspectives." (Dai et al., 2023a, discussing gradient descent analogy, p. 1113)

"All these warmup methods improve the ICL capability by updating the model parameters, which implies that the ICL capability of the original LLMs has great potential for improvement." (Takeaway on training, p. 1124)

"Format matters more than label correctness" (Paraphrase of Min et al. findings, Section 5.1.2)

"ICL heavily relies on high-quality demonstrations selected from annotated examples, which are often scarce in low-resource languages and tasks." (Challenge: Generalization, p. 1114)

AIWG Implementation Mapping

Direct Application to AIWG Templates

ICL ElementAIWG ImplementationLocation
DemonstrationsTemplate examples (2-4 examples)All templates in `sdlc-complete/templates/`
InstructionTemplate purpose/headerTemplate front matter
Format SpecificationTemplate structure (sections, fields)Markdown structure in templates
Few-shot2-3 examples per templateStandard across AIWG
Consistent StructureIdentical format across examplesVoice templates, use case templates
Relevant SelectionExamples similar to target taskDomain-specific template examples
OrderingSimple to complex when applicableTutorial sequences

Template Design Guidelines from ICL Research

1. Prioritize Format Consistency (from Finding 1):

# AIWG Template Pattern

## Example 1: [Simple Case]
**Field A**: Value
**Field B**: Value
**Outcome**: Result

## Example 2: [Moderate Case]
**Field A**: Value  # Same structure
**Field B**: Value  # Same structure
**Outcome**: Result

## Your Task
**Field A**: [USER INPUT]
**Field B**: [USER INPUT]
**Outcome**: ?

Rationale: Format consistency matters more than individual example perfection. AIWG templates should maintain identical structure across all examples.

2. Demonstration Selection Strategy (from Section 4.1.1):

  • Use k-NN for domain relevance: Select examples semantically similar to likely use cases
  • 2-4 demonstrations optimal: Survey shows few-shot (k=1-10) typical; AIWG uses 2-4
  • Diversity regularization: Avoid redundant examples; ensure variety in complexity

3. Simple-to-Complex Ordering (from ICCL, Liu et al., 2024b):

## Example 1: Basic Use Case
[Simplest possible instance]

## Example 2: Intermediate Use Case
[More fields, moderate complexity]

## Example 3: Advanced Use Case
[Full complexity, edge cases]

4. Instruction Clarity (from Section 4.2):

  • Include explicit task instruction at template header
  • Natural language description of what to create
  • Specify expected output format

5. Chain-of-Thought for Complex Templates (from Section 4.2):

## Example with Reasoning Steps

**Requirement**: User needs secure authentication

**Analysis**:
1. Identify actors (User, System)
2. Determine preconditions (valid credentials exist)
3. Map main flow steps
4. Consider error cases

**Output**: [Complete use case]

Agent System Prompts and ICL

Few-Shot Agent Prompts (`sdlc-complete/agents/`):

# Test Engineer Agent

## Your Role
You are a test engineer specializing in test strategy and test case design.

## Example 1: Unit Test Strategy
**Given**: User authentication module
**Test Approach**:
- Test valid credentials → expect success
- Test invalid credentials → expect failure
- Test edge cases (empty, special characters)
**Output**: [Detailed test cases]

## Example 2: Integration Test Strategy
[Similar structure...]

## Your Task
Given the following requirement, create a test strategy...

Application: Agent definitions use ICL patterns with 1-3 examples to establish behavior.

Prompt Engineering Principles for AIWG

From ICL Research → AIWG Practice:

ICL PrincipleAIWG Application
Format > ContentTemplate structure is rigid; content flexible
Order MattersSimple examples first in tutorials
Warmup HelpsAgent "warmup" via system prompt examples
DiversityExamples cover different use case types
RelevanceExamples from same domain as target
Minimal Set2-4 examples, not 10+

Scoring Functions for AIWG Validation

Not directly applicable (AIWG generates documents, not classifications), but conceptually:

  • Direct: Validate template compliance (structural checks)
  • PPL: Could use perplexity to detect off-topic content
  • Channel: Reverse validation (given output, does it match input requirements?)

Integration with AIWG Workflows

1. Template Creation Workflow

ICL-Informed Process: 1. Define task instruction (clear, natural language) 2. Select 2-4 representative examples (diverse but relevant) 3. Order simple → complex 4. Ensure identical structure across examples 5. Test with minimal set (don't over-provide examples)

2. Agent Prompt Engineering

Few-Shot Agent Design:

# [Agent Name]

You are a [role] specializing in [domain].

## Example Interaction 1
**User Request**: [Simple request]
**Your Response**: [Ideal response showing format, tone, content]

## Example Interaction 2
**User Request**: [More complex request]
**Your Response**: [Ideal response with reasoning steps]

## Now Handle
**User Request**: [Actual user input]

Rationale: Agents learn behavior from demonstration examples (ICL), not just instruction.

3. Quality Gate Checklists

ICL-Inspired Checklist Pattern:

# Security Review Gate

## Example 1: Authentication Review
- [ ] Verify authentication mechanism documented
- [ ] Check credential storage security
- [ ] Validate session management
**Result**: PASS

## Example 2: Data Protection Review
[Similar checklist...]
**Result**: PASS with conditions

## Your Review
[Apply same checklist to your artifact]

Challenges and Future Directions (from Section 7)

1. Efficiency and Scalability

Challenge: "The use of demonstrations in ICL introduces two challenges: (1) higher computational costs with an increasing number of demonstrations (efficiency), and (2) fewer learnable samples due to the maximum input length of LLMs (scalability)." (p. 1114)

AIWG Consideration: Templates should remain concise; avoid bloating with excessive examples.

2. Generalization to Low-Resource Scenarios

Challenge: "ICL heavily relies on high-quality demonstrations selected from annotated examples, which are often scarce in low-resource languages and tasks." (p. 1114)

AIWG Relevance: Cross-task ICL could enable templates designed for one domain (e.g., web apps) to adapt to another (e.g., embedded systems) with minimal examples.

3. Long-Context ICL

Challenge: "Researchers have found that increasing the number of demonstrations does not necessarily enhance performance and may even be detrimental." (p. 1115)

AIWG Practice: Current practice of 2-4 examples appears optimal; avoid template bloat.

4. Understanding ICL Mechanisms

Challenge: "Despite preliminary explanations (Dai et al., 2023a; Jiang, 2023), the underlying working mechanism of ICL remains unclear and requires further investigation." (p. 1108)

Research Opportunity: Better mechanistic understanding could improve AIWG template design.

Cross-References

  • REF-025: Constitutional AI - Self-critique patterns (similar to ICL's iterative improvement)
  • REF-016: Chain-of-Thought Prompting - CoT as ICL application
  • REF-006: Sweller's Cognitive Load Theory - Worked examples parallel to ICL demonstrations

Foundational ICL:

  • Brown et al. (2020) - GPT-3: Few-Shot Learners (ICL emergence)
  • Wei et al. (2022c) - Chain-of-Thought Prompting (reasoning ICL)
  • Min et al. (2022c) - Rethinking Demonstrations (format importance)

Demonstration Selection:

  • Liu et al. (2022) - KATE: k-NN retrieval for ICL
  • Rubin et al. (2022) - EPR: Supervised retrieval
  • Su et al. (2023) - Votek: Graph-based diverse selection

Theoretical Understanding:

  • Dai et al. (2023a) - ICL as Gradient Descent
  • Xie et al. (2022) - ICL as Bayesian Inference
  • Olsson et al. (2022) - Induction Heads mechanism

Training Enhancements:

  • Wei et al. (2022a) - FLAN: Instruction tuning
  • Min et al. (2022b) - MetaICL: Meta-learning for ICL
  • Wei et al. (2023a) - Symbol Tuning: Decouple format from semantics

Quick Reference: Topic → Section/Page

TopicSectionPage(s)
ICL Definition§2 Definition and Formulation1108-1109
Demonstration Selection§4.1.11110
Demonstration Reformatting§4.1.21110-1111
Demonstration Ordering§4.1.31111
Instruction Formatting§4.21111-1112
Scoring Functions§4.31112
Pretraining Factors§5.1.11112
Inference Factors§5.1.21113
Induction Heads§5.2.11113
Bayesian Interpretation§5.2.21113
Gradient Descent View§5.2.21113-1114
Applications§61114
Challenges§71114-1115
Experimental ComparisonAppendix B1125-1126
TakeawaysAppendix A1124-1125
Beyond Text (Vision, Speech)Appendix D1127-1128

Methodological Details

Taxonomy Construction

Approach: Literature review with hierarchical categorization (Figure 2, p. 1108)

Three Dimensions: 1. Training: Pre-training, Warmup 2. Inference: Demonstration (selection, reformatting, ordering), Instruction, Scoring 3. Analysis: Influencing Factors (pretraining stage, inference stage), Learning Mechanism (functional modules, theoretical interpretation)

Experimental Setup (Appendix B)

Models Tested: GPT-2, GPT-J, Llama3-8B-Instruct, Qwen2-7B-Instruct

Datasets: SST-2, SST-5, CommonsenseQA, AG News, SNLI

Configuration:

  • 8 demonstrations per task
  • 1000 training examples for retrieval
  • 1000 test examples for evaluation
  • PPL-based scoring function
  • Framework: OpenICL

Metrics:

  • Accuracy: Classification performance
  • Efficiency: Inference latency (ms)
  • Stability: Variance across 5 random seeds

Limitations and Scope

Survey Limitations (from Section 8, p. 1115):

1. Coverage: "Given the extensive body of related work, particularly in demonstration design and the principle analysis of ICL, we may have overlooked some equally valuable contributions."

2. Recency: "Many papers covered by this survey did not utilize the most up-to-date models while running experiments."

3. Scope: Survey focuses on NLP; visual/speech ICL covered briefly in Appendix D

Out of Scope:

  • Detailed prompt engineering beyond ICL
  • Full coverage of multimodal ICL
  • Production deployment considerations
  • Cost-benefit analysis of ICL vs fine-tuning

Revision History

DateAuthorChanges
2026-01-24Research Acquisition (#74)Initial comprehensive reference entry

Notes for AIWG Implementation

High-Priority Takeaways

1. Template Design: Use 2-4 examples with identical structure (format > content) 2. Example Ordering: Simple to complex when applicable 3. Agent Prompts: Few-shot examples in system prompts establish behavior 4. Instruction Clarity: Explicit task descriptions at template headers 5. Demonstration Selection: Choose examples relevant to target domain

Medium-Priority Considerations

1. Warmup for Agents: Could pre-tune agent models on AIWG-specific tasks 2. CoT in Complex Templates: Add reasoning steps for architecture decisions, risk analysis 3. Scoring Functions: Consider template validation approaches inspired by ICL scoring 4. Long-Context Trade-offs: Monitor when adding more examples hurts vs. helps

Low-Priority (Research Directions)

1. Meta-Learning: Could AIWG learn optimal template structures from usage data? 2. Automatic Template Generation: Use LLMs to generate templates from task descriptions 3. Cross-Task Transfer: Can templates designed for one domain adapt to others? 4. Theoretical Understanding: Apply gradient descent or Bayesian interpretations to understand why AIWG templates work

Open Questions for AIWG

1. Optimal k for AIWG: Is 2-4 examples optimal, or should some templates have more? 2. Domain-Specific vs. Generic: Should templates be highly domain-specific or more generic? 3. Example Diversity: How to balance similarity (relevance) with diversity (coverage)? 4. Instruction vs. Demonstration: When to rely on explicit instructions vs. implicit examples?

Further Reading

If interested in...

  • Demonstration Selection: Liu et al. (2022) KATE, Rubin et al. (2022) EPR
  • Theoretical Mechanisms: Dai et al. (2023a) Gradient Descent, Xie et al. (2022) Bayesian
  • Chain-of-Thought: Wei et al. (2022c), Zhang et al. (2023c)
  • Instruction Tuning: Wei et al. (2022a) FLAN, Chung et al. (2022) Scaling Instructions
  • Practical Prompting: Li (2023) Practical Survey on Prompt Design
  • Retrieved Demonstrations: Xu et al. (2024) Survey on Retrieval for ICL