Loading slide

The Representation Problem

  1. 01The Representation Problem
  2. 02The gap between pixels and words
  3. 03How a lesson spreads
  4. 04Why the numbers have to mean something
  5. 05Go on, try it
  6. 06So add a direction
  7. 07One-hot encoding
  8. 08The question that breaks it
  9. 09What we actually need
  10. 10You already do this
  11. 11Meaning is relational
  12. 12Reinforce your understanding
  13. 13Question: Why labels fail
  14. 14Question: Meaning from company
  15. 15Question: What good numbers do
  16. 16Quiz: answer
  17. 17What to carry out of this chapter
  18. 18Want to go deeper?
3 / 17
BackNext

How a lesson spreads

To see why that broken link is fatal, we need one fact about how a network behaves.

Back in the learning module we called it generalisation: a trained network can handle a dog photo it has never seen before, not just the ones it studied. We saw that it happens. Now we need to look at how, because the how is the thing words are about to break.

A network treats similar inputs similarly. Feed it two photos whose numbers are nearly the same, and it answers nearly the same about both.

It cannot really do otherwise, because of what a network is made of. Remember the dials from the learning module: a network is a vast bank of them wired together. The answer is what comes out once a set of numbers has been turned through all those dials.

Nudge the numbers going in by a hair and you have barely moved any dial, so what comes out can only shift by a hair too. There is no hidden trapdoor that flips the answer from "almost certainly a dog" to "definitely not a dog" over a change too small to see.

Small change in, small change out. That gentleness is generalisation.

Now make that concrete with numbers on a line. Picture inputs laid out like house numbers along a street: 1, 2, 3, 4.

The network learns something true about whatever sits at number 3, say "things around here tend to bark." Because of that gentleness, it does not keep the lesson pinned to exactly 3. It bleeds across to the immediate neighbours, 2 and 4, a little less to 1 and 5, fading with distance.

That bleed is the same generalisation, just viewed up close: near in number, treated alike.

That is a gift for images. With pixels, near in number really does mean near in reality. A brightness of 128 and one of 130 are nearly the same grey. So when the lesson bleeds from 128 to 130, it lands on a genuinely similar shade and helps.

Hold onto this one idea: a network learns about one number and automatically bleeds that lesson to its neighbours.

The next slide hands the network word-numbers, where the neighbours are total strangers, and watches that gift turn into a curse.

Citations(2)↓
  1. 1. cs.brown.edu
  2. 2. ibm.com
Citations(2)↓
  1. 1. cs.brown.edu
  2. 2. ibm.com