From Fragment Shaders to Tensor Cores: How CUDA Actually Works ?

Your system has a CPU. It probably also has a GPU. They look similar on a spec sheet - both have "cores," both run at some clock speed, but they were built to solve completely different problems. Understanding why is the key that unlocks everything else about CUDA. Why does this hardware even exist? A CPU is latency-optimized. Its job is to finish one complex task as fast as possible, so a large fraction of its transistor budget goes toward things that have nothing to do with raw arithmetic:…