Skip to content
GetProfitable
Search
Dictionary

Gradient boosting

An ensemble that fits trees sequentially, each correcting the previous ones' errors. Powerful on tabular data, and correspondingly easy to overfit on market data.

Boosting usually outperforms a random-forest on clean tabular problems, which is why it dominates data-science competitions. That success does not transfer automatically: competition datasets have strong signal and independent rows, and financial datasets have neither.

It needs careful control. Low learning rate, shallow trees, strong subsampling, early stopping on a properly purged validation fold, and a hard cap on the number of features. Without these it will find whatever idiosyncrasies exist in your particular decade of history.

Sensible use: as a comparison against a regularised linear model on the same features. If boosting adds only a little, take the linear model, because you will be able to understand why it fails when it does.

Related: random-forest, regularisation, hyperparameter, overfitting

Educational only, not advice. Spotted an error? Post in Site Feedback.