CLIP learns from captions, not labels — two encoders, one shared space, and an N N matrix whose diagonal must win

Train an image classifier the usual way and you're locked to a fixed list of labels. CLIP throws that out. Instead of labels it learns from captions — the raw (image, caption) pairs the web already has by the hundreds of millions (about 400M) — by training two encoders at once so that a picture and its caption land at the same point in one shared space. No hand-labelled classes at all: the caption is the supervision. Here's the idea, built up from the pieces. Two encoders into one space There's…