Softmax and cross-entropy: the clean p y gradient that trains every classifier

Softmax and cross-entropy are almost always taught together, and there is a beautiful reason for it that a lot of tutorials skip over. Individually they are a little fiddly — softmax has a full Jacobian, cross-entropy has a 1/p in its derivative. But fuse them and the entire gradient collapses to one subtraction. I built an interactive page where you drag four logits and watch the probabilities, the loss, and that gradient update live, and the cancellation is the whole point. Logits are not…