Skip to content
GetProfitable
Search
Dictionary

Model risk

The risk that your risk numbers are wrong because the model behind them assumes a world that does not exist.

Every risk figure is the output of assumptions: normal returns, stable correlations, continuous prices, available liquidity. Each of those is false in the conditions that produce large losses, which means model error is correlated with the events the model is supposed to warn you about.

History supplies the examples, and they share a shape. Positions sized by a model, a regime change the model had never seen, and losses many multiples of the stated worst case. The failure was not arithmetic; it was believing a number computed from a period that did not contain the event.

Defences are unglamorous. Compute every important number two ways and compare. Treat any estimate built on fewer than a few hundred independent observations as a rough direction. Size so that being wrong by a factor of three is survivable, because on the day it matters you will be. See fat-tails and sample-size-for-edge.

Related: fat-tails, value-at-risk, sample-size-for-edge, stress-testing

See it drawn

Original diagrams for the ideas on this page. Illustrative, not real market data.

The spread of outcomes behind an expectancyA histogram of forty trades: a tall block of small losses on the left, a low spread of larger wins on the right, and a line marking the average outcome.NUMBER OF TRADES051024 LOSSES, AVG −$20016 WINS, AVG +$600EXPECTANCY +$120−$400−$200$0+$200+$400+$600+$800PROFIT OR LOSS PER TRADEexpectancy = (40% × $600) − (60% × $200) = +$120 per trade
Expectancy: the average trade. Forty trades sorted by outcome: 24 small losses and 16 larger wins. Weighting each side by how often it happens gives the average result per trade, marked here by the dashed line at +$120.

Educational only, not advice. Spotted an error? Post in Site Feedback.