Loading slide

What a Trained Model Actually Is

  1. 01What a Trained Model Actually Is
  2. 02Memory card
  3. 03A model is a file
  4. 04Two files
  5. 05How the two fit together
  6. 06A Piano
  7. 07The valuable file
  8. 08You connect to it
  9. 09Where does training data come from?
  10. 10Cleaning the data
  11. 11The network can't read
  12. 12Guess the next word
  13. 13Right or wrong
  14. 14One nudge, a billion times
  15. 15Take a breath
  16. 16Where your memories live
  17. 17A base model is not an assistant
  18. 18Why it's called pre-training
  19. 19The weights are the model
  20. 20The scale of it
  21. 21Why so many?
  22. 22What a weight actually is
  23. 23Everywhere and nowhere
  24. 24Using the model: inference
  25. 25The whole arrangement
  26. 26Reinforce your understanding
  27. 27Question: What is a model, really?
  28. 28Question: Learning from raw text
  29. 29Question: The model doesn't know what it doesn't know
  30. 30Quiz: answer
  31. 31The machine learned something real.
  32. 32Want to go deeper?
5 / 31
BackNext

How the two fit together

How does the program know what to do with that giant pile of numbers? The answer is simpler than you'd expect.

In code, a network is plainer than people imagine. Stripped of the technical wording, it just lays out a shape: "first layer, two numbers in, three out. Next layer, three in, one out." And the moment you write that shape down, the number of weights is fixed. The program never lists the weights themselves. It only sets up empty slots, like an egg carton with a known number of cups, in a fixed order.

The weights file is the other half: just numbers, one after another, no labels. It doesn't need any. To load the model, the program reads the file from the top and drops each number into the next slot, in order. Press the button and watch.

The order is the address. That's the whole trick.

Which is why the two files are a matched pair. Change the shape by a single neuron and the numbers no longer line up. Too many, or too few, and the model simply refuses to run. The weights are meaningless without the shape that reads them. The shape is empty without the weights.

Citations(2)↓
  1. 1. docs.pytorch.org
  2. 2. github.com
Citations(2)↓
  1. 1. docs.pytorch.org
  2. 2. github.com