Tokens

  1. 01Tokens
  2. 02Not words. Tokens.
  3. 03What the model actually sees
  4. 04Context windows are measured in tokens
  5. 05Reinforce your understanding
  6. 06Question: Why not just use words?
  7. 07Question: Weird splits
  8. 08Quiz: answer
  9. 09Tokens are the unit of everything
  10. 10Want to go deeper?
BackHomeNext chapter