Loading slide

The Representation Problem

  1. 01The Representation Problem
  2. 02The gap between pixels and words
  3. 03How a lesson spreads
  4. 04Why the numbers have to mean something
  5. 05Go on, try it
  6. 06So add a direction
  7. 07One-hot encoding
  8. 08The question that breaks it
  9. 09What we actually need
  10. 10You already do this
  11. 11Meaning is relational
  12. 12Reinforce your understanding
  13. 13Question: Why labels fail
  14. 14Question: Meaning from company
  15. 15Question: What good numbers do
  16. 16Quiz: answer
  17. 17What to carry out of this chapter
  18. 18Want to go deeper?
2 / 17
BackNext

The gap between pixels and words

Let's start from the one fact this whole module hangs on: a neural network cannot see, read, or hear anything. It only ever works with numbers. So before it can learn from a thing, that thing has to be turned into numbers first. For images, that turned out to be easy. For words, it turned out to be the hard problem of the decade. This slide is about why those two cases are so different.

Start with images, because there the numbers come for free. A digital photo is made of tiny dots called pixels, and each pixel is stored as a number for how bright it is: a low number for a dark dot, a high number for a bright one. Pure black might be 0, pure white 255, a mid-grey something around 128. An ordinary photo is just a giant grid of these brightness numbers, around 150,000 of them, and the network can start computing on them straight away.

Here is the part that matters, and it is easy to skim past. Those numbers don't just label the pixels; they genuinely describe them. Two dots that look almost the same shade of grey really do get two almost-equal numbers, say 128 and 130. So the closeness of the numbers is not a coincidence: numbers that sit near each other belong to dots that look alike. The arithmetic lines up with the real world. That single property, near in number means near in reality, is the quiet reason deep learning cracked images first.

Words do not come with numbers like that, and any we invent ourselves lack that magic property.

Suppose we just number the dictionary: "a" is 1, "the" is 2, "dog" is 3, "dollar" is 4, and on down the list. Now look at what we have accidentally claimed. "Dog" got 3 and "dollar" got 4, so by the only thing the network can read, the numbers, they are right next to each other, practically the same. Meanwhile "puppy," sitting far down the alphabet, gets some number like 6,000, miles away from "dog." We have placed "dog" next to "dollar" and far from "puppy," which is exactly backwards. The number is just a name tag, handed out by alphabetical accident. It says nothing real about what the word means.

With pixels, closeness of numbers meant closeness in reality, and the network could trust it. With these word-numbers, closeness means nothing at all. That broken link is the entire problem, and the next slide shows why it is fatal rather than merely untidy.

Citations(2)↓
  1. 1. cs.brown.edu
  2. 2. ibm.com
Citations(2)↓
  1. 1. cs.brown.edu
  2. 2. ibm.com