Hinge Regression Trees and HRT-Boost: Newton-Optimized Oblique Learning for Compact Tabular Models

Researchers introduced Hinge Regression Tree (HRT), a new oblique decision tree method for tabular data trained as a nonlinear least-squares problem with damped Newton updates.

They also proposed HRT-Boost, a stage-wise boosting ensemble built on HRT to improve predictive performance while keeping models compact.

The paper includes approximation and optimization guarantees, giving the method a stronger theoretical foundation than many tree-based approaches.

On synthetic and real-world benchmarks, the methods show strong results and are positioned as competitive options for small tabular ML models.