A Test for Superior Predictive Ability
Read the paperopens doi.org in a new tab
What they found
Hansen improved White's reality check, which loses power when the set of tested models includes many that are clearly poor (adding bad models makes it harder for a good one to pass). The SPA test re-centers the null distribution so that obviously inferior models do not dilute the test, giving a more powerful and less manipulable check on whether the best model in a set has real predictive ability. It has become the standard tool for data-snooping-robust evaluation of trading rules and forecasting models.
What you can use
- The SPA test is the modern standard for asking whether the best of many strategies is genuinely good, and it is harder to game than its predecessor.
- Padding your universe with junk strategies to make the winner look better does not work with this test.
- Most published intraday technical-rule studies that use the SPA test find no surviving rules.
Caveats
Econometrics paper; implementation requires bootstrapping the full set of models. Still assumes the tested set is the full set that was tried.
Tags: backtesting, data-snooping, statistics, forecasting
Summaries are our own reading of the paper, not the authors' words. Educational only, not advice. Discuss it in Book Club.