Loading slide
If 256 bytes can spell everything, a sharp question follows: why bother with the messy tokenizer at all? Why not feed the model nothing but raw bytes and skip the whole business of vocabularies and weird splits?
You can. Some real models do exactly this; they are called byte-level models, and they work. So your instinct is right: the bytes are genuinely enough.
That makes the question sharper, not softer. If bytes are enough, why do many large language models still use tokens?
To see it, we have to look ahead to where meaning actually lives. In the next chapter, each token gets an
So how could a byte-only model ever know "cat" is like "tiger," if the letters "c", "a", "t" mean nothing on their own?
The answer is the machine from the attention chapter. Attention lets pieces look at their neighbours and combine into something richer than any one of them. A byte-level model leans on this entirely: it sees "c", "a", "t" sitting in a row and, through layer after layer of attention, assembles a cat-meaning out of letters that individually meant nothing. The meaning is not in any letter. It emerges from the letters being combined, the same way (remember the going-deeper module) wetness emerges from water molecules, none of which is itself wet.
Now the two paths stand clearly side by side, and this is the whole point of the chapter.
A token model stores meaning. "cat" is one token, one lookup, and its embedding is already sitting next to "tiger." The meaning arrives pre-made.
A byte model assembles meaning. It must rebuild "cat" from near-meaningless letters every single time, using up a big share of its attention and its layers just to re-spell words before it can even start thinking about them.
Both reach "cat is like tiger." Tokens arrive there cheaply; bytes arrive there the hard way. That is the trade. Tokens hand the model pre-made word-meanings so its effort goes into reasoning instead of spelling, at the cost of a fiddly tokenizer and all its quirks. Bytes are beautifully simple and perfectly fair across languages, at the cost of making the model do far more work to reach the same understanding (and far longer sequences, which eat the context window we will meet later).
Many large models pick tokens, because that pre-made meaning is an enormous saving. But byte models are a real alternative, and as computers get cheaper, "just feed it bytes and let attention figure it out" keeps getting more tempting.
One last thing, so a stray worry does not follow you out. That assembling is a skill the model learns during training, when its weights are slowly shaped. It then performs the assembling fresh for your sentence at the moment you chat with it, weights frozen. Training builds the ability; using it puts the ability to work.