Your LLM fused the two columns you asked for — and the eval marked it wrong

You ask a text-to-SQL model to "list the members' names". The benchmark's gold query returns first_name, last_name — two columns. The model returns one: a helpfully assembled full name. Read side by side, the model's answer is arguably the better one. The scorer marks it wrong, every single time, and your engine number reads lower than the engine is. -- gold: two columns SELECT first_name , last_name FROM member WHERE ...; -- model: one column, same information SELECT first_name || ' ' ||…