Loading slide

The Transformer

  1. 01The Transformer
  2. 02Memory card
  3. 03Attention is all you need
  4. 04A stamp for position
  5. 05Take a breath
  6. 06Two ways to use it
  7. 07Train once, adapt anywhere
  8. 08Why it took over everything
  9. 09Reinforce your understanding
  10. 10Question: Why transformers took over
  11. 11Question: Pretraining and finetuning
  12. 12Question: Where word order went
  13. 13Quiz: answer
  14. 14The architecture everything is built on
  15. 15Want to go deeper?
5 / 14
BackNext

Take a breath

The machine is built. Before we put it to work, here it is in one breath.

A transformer is a tower of attention. Words come in at the bottom, each stamped with where it sits. Floor by floor, every word reaches out, weighs every other word, and blends in what matters, coming out a little more aware of its context each time. Stack enough floors and the model builds up meaning in stages, the same edges-to-objects shape we keep meeting, now aimed at language.

That is the whole architecture. Attention, stacked, with a stamp for word order. No single-file reading anywhere in it.

What is left is not more machinery. It is the two questions that decide what a transformer actually becomes: how do you use it, and how do you train it? Those two answers are what turned this one architecture into translators, search engines, and the chatbots you have talked to. We take them next.

Citations