Loading slide
Here is a way to feel what tokenization does. Imagine replacing every token in a sentence with a unique little symbol.
"The cat sat on the mat" might become: 🌀 🎯 🍄 🔮 🌀 🎲
That is closer to what the model receives. Not words. A row of symbols, each one a stand-in for a chunk of text, each with its own ID number underneath. The model has no idea these symbols were ever language. To it, they are just abstract markers.
Notice that "the" shows up twice and gets the same symbol both times. That consistency is the point: every time that fragment appears, anywhere in any text, it becomes the exact same token with the exact same ID. The model's entire world is built from a fixed alphabet of these markers, tens of thousands of them, and nothing else.
Hold onto this picture. What the model does with these symbols, how it learns which ones tend to follow which, is the heart of a later module. For now, the key shift is this: the words you see and the symbols the model sees are not the same thing.