XGBoost: The Accountant Who Read the Tax Code So Carefully He Found Money Nobody Else Saw

The One-Line Summary: XGBoost is gradient boosting with three changes that each look like a footnote and together won a decade of competitions — it uses the second derivative so each leaf knows how stiff its correction is rather than only how wrong, it writes the cost of every new leaf directly into the objective so pruning becomes arithmetic instead of a max_depth guess, and it treats a missing value as a direction to be learned rather than a hole to be filled. The Parable of the Counting…