Temperature and sampling

Sampling parameters decide how the next token is picked. Low temperature for extraction and classification, higher for anything that should feel varied.

Lesson 1.28m 11sIntermediate25k students

Chapters

8:11
  1. 0:00Intro
  2. 0:37Greedy Decoding
  3. 1:05Random Sampling
  4. 1:50Temperature
  5. 3:55Top-k Sampling
  6. 4:27Top-p Sampling
  7. 5:10Pros and Cons
  8. 7:30Outro

What this lesson covers

  • Trade determinism against variety with temperature
  • Know what top-p changes and when to touch it
  • Pick settings per task rather than globally

This lesson sits in LLM Fundamentals, part of Building AI Apps with LLMs. It assumes what came before it and leads directly into the next lesson in the module.

In this lesson you will

  • Trade determinism against variety with temperature
  • Know what top-p changes and when to touch it
  • Pick settings per task rather than globally

Resources