Evaluating Trading Strategies
Read the paperopens papers.ssrn.com in a new tab
What they found
A practitioner-oriented guide to the multiple-testing problem. Harvey and Liu explain, with examples, why a strategy's t-statistic must be adjusted for the number of strategies tried, present the standard corrections (Bonferroni, Holm, and false-discovery-rate methods), and translate them into a 'haircut' that shrinks a backtested Sharpe ratio to what should be expected given the search. They show that a Sharpe ratio of 1.0 found after trying 100 strategies might warrant a haircut of 50% or more, and provide a rule of thumb that a strategy needs a t-statistic around 3 to be taken seriously.
What you can use
- A backtested Sharpe ratio should be cut, sometimes in half, before you believe it, and the cut depends on how many things you tested.
- The paper gives a usable procedure for the haircut, not just a warning.
- A t-statistic of 3, not 2, is the practical bar for a strategy that came out of a search.
Caveats
Corrections depend on assumptions about how correlated the tested strategies are. Written for practitioners; the companion paper 'Backtesting' is more technical.
Tags: backtesting, multiple-testing, sharpe-ratio, beginner-friendly
Summaries are our own reading of the paper, not the authors' words. Educational only, not advice. Discuss it in Book Club.