Adding a Third Kind of Judge: What Cross-LLM Evaluation Changed About the Winner

1Key Takeaways

• Part 3 of a series on auto-grooming Jira backlogs with ML and LLMs.
• Read Part 1 and Part 2 for the original pipeline and the first Gemini-vs-human evaluation.
• Picking Up Where Part 2 Left Off Part 2 compared two clustering pipelines through two LLMs, judged by Gemini, then checked against my own review.
• The finding that stuck with me: Gemini kept flagging correct arithmetic as hallucination, specifically on outputs where a model had to compute something rather than just quote it back.