Prefill-Decode Disaggregation: When Splitting LLM Inference Pays

1Key Takeaways

• Originally published on AI Tech Connect .
• What you need to know Prefill — the phase that reads your prompt and produces the first token — is a set of large matrix multiplications over every prompt token at once.
• It saturates a GPU's floating-point units.
• Decode — the phase that emits every token after the first — processes exactly one token per step per sequence, and spends most of its time re-reading the key-value cache out of high-bandwidth memory.