Skip to content
GetProfitable
Search
Dictionary

Combinatorial purged cross-validation

A validation scheme that tests many different combinations of held-out blocks, producing a distribution of backtest results instead of a single path.

Split the history into N blocks, then choose every combination of k blocks as the test set. With 10 blocks and 2 held out, you get 45 combinations and can assemble multiple distinct out-of-sample paths rather than one.

The output is the point. Instead of a single Sharpe of 1.1 you get 45 numbers whose distribution you can inspect: a median of 1.0 with a 5th percentile of 0.2 is a very different claim from a median of 1.0 with a 5th percentile of minus 0.8.

It is expensive in compute and still requires purging and an embargo-period at every boundary. It also assumes the blocks are interchangeable, which is a strong assumption across a regime shift such as 2020.

Related: purged-cross-validation, cross-validation, monte-carlo-simulation, probability-of-backtest-overfitting

Educational only, not advice. Spotted an error? Post in Site Feedback.