Sparse Autoencoders and the Quest to Find 'Concepts' Inside Models

You look inside a neural network. You see billions of numbers. They are not words. They are not images. They are not concepts. They are just weights. You know the model can recognize a cat. You know it can generate a poem. But you cannot find the "cat" in the numbers. It is not there. It is distributed. It is invisible. This is the black box problem. We built the machine. We trained the machine. But we cannot read its mind. Now, a new tool is emerging. Sparse autoencoders are helping us peek…