Stochastic Sampling is Epistemically Shallow: The Dimensionality Gap Between Temperature Variation and Model Diversity in LLMs

1Key Takeaways

• arXiv:2607.20464v1 Announce Type: new Abstract: When a language model gives different answers on repeated runs, does that variation reveal what it does not know?
• Self-consistency turns the variation into a per-question uncertainty estimate via majority voting.
• But does the same variation reveal cross-question structure -- related questions flipping together, the way a diverse ensemble does?
• We compare two regimes on the same questions: one model run $100$ times at $\tau=1$ versus an ensemble of….