Loading slide

Next Token, Every Time

  1. 01Next Token, Every Time
  2. 02Memory card
  3. 03One word at a time
  4. 04Painting into a corner
  5. 05Temperature: how random is random?
  6. 06Turn the dial yourself
  7. 07Take a breath
  8. 08No inner monologue
  9. 09Thinking out loud actually helps
  10. 10Reinforce your understanding
  11. 11Question: No plan
  12. 12Question: Why reasoning out loud helps
  13. 13Quiz: answer
  14. 14The output is the thinking
  15. 15Want to go deeper?
6 / 14
BackNext

Turn the dial yourself

Here is the dial, in your hands. The phrase "The cat ..." is waiting for its next word, and the model has scored a handful of candidates. Each bar is how likely the model thinks that word is to come next. "Sat" is the safe favourite; "exploded" is desperate but not quite impossible.

Slide the temperature from cold to hot and watch the bars change shape. Cold, and one bar towers over the rest: the model will almost always say "sat." Hot, and the bars flatten toward each other, so the rare words suddenly have a real chance. Nothing about the words changed. Only how much the model is willing to gamble.

Then press "pick a word" a few times at each setting. Cold, you get "sat" almost every time. Warm, you get "sat" or "slept" or "ran," sensible variety. Hot, and "yawned" or even "exploded" can win. That is the same randomness that makes a model give you two different answers to one question, made into something you can feel turning under your finger.

Citations
Citations