CatBoost beats target leakage with order: ordered target statistics, ordered boosting, and symmetric trees

1Key Takeaways

• XGBoost gave gradient boosting a rigorous second-order objective; LightGBM made the same boosting fast.
• CatBoost completes the trio by attacking a subtler enemy: target leakage .
• It keeps the exact g, h gradient-boosting engine, but obsesses over two places where the label quietly leaks into training — and its fix, for both, is the same one word: ordering .
• I built a from-scratch demo that computes all of this live in JS, no library, so you can watch the leak open and close as you move sliders.….