Autoscaling LLM Inference on Kubernetes with KServe and KEDA

1Key Takeaways

• Originally published on AI Tech Connect .
• What you need to know Kubernetes has quietly become the default place teams run LLM inference.
• According to CNCF's 2025 Annual Cloud Native Survey, 66% of organisations hosting generative AI models use Kubernetes to manage some or all of their inference workloads — the majority, and rising.
• That matters because the autoscaler most teams reach for first, the standard Horizontal Pod Autoscaler wired to CPU and memory, is built for the wrong bottleneck.