Skip to content
GetProfitable
Search

Monitoring divergence and when to stop

Lesson 28 · about 11 min

Every system that is traded live will eventually go through a period where it does worse than the backtest. Most of those periods are ordinary variance: the drawdowns that Monte Carlo predicted, arriving on schedule. Some are the edge disappearing. The difference between a trader who survives and one who does not is often whether they decided, in advance, how to tell the two apart, and what to do about each.

What to track

Track the live results against the backtest on a small set of numbers, updated after every trade:

Metric Backtest value Live value (running) Alarm level
Trades per month 5.0 4.6 Below 3.5 or above 6.5
Win rate 41% 38% Below 30% over 50 trades
Average R +0.22 +0.15 Below 0 over 50 trades
Slippage per trade (R) 0.06 0.09 Above 0.12
Current drawdown (R) 8.4 Above 18.2 (MC 95th)
Longest time underwater 11 months 4 months Above 14 months
Rule breaches 0 1 Any pattern

The alarm levels are set from the backtest's own variability, using the standard error from Module 5 for the averages and the Monte Carlo distribution for drawdown. They are written down before going live. Nothing about them changes because a month felt bad.

Ordinary variance versus a broken edge

With a +0.22R expectancy and a 1.6R standard deviation, the standard error over 50 trades is 1.6 ÷ √50 = 0.23R. A 50-trade live average anywhere between −0.24R and +0.68R is within two standard errors of the backtest. That is a wide band, and it means that 50 losing-ish trades do not, on their own, prove anything. What they do is keep the drawdown monitor honest: if the trades are within variance but the drawdown has crossed the 95th percentile, the drawdown rule fires regardless.

Signs that point to a broken edge rather than variance:

  • Trade frequency has changed materially. The market is producing different conditions from the ones the rules were built on.
  • Slippage has risen and stayed risen. The mechanism may be getting crowded, or liquidity has changed.
  • The mechanism you wrote down in Module 1 no longer applies: the participants have changed, the product has changed, the regulation has changed.
  • Losses are clustered in a condition the backtest rarely contained.

Signs that point to variance:

  • Frequency, slippage and win rate are in range; only the average R is down, driven by a few larger-than-usual losses.
  • The drawdown is inside the Monte Carlo distribution.
  • The equity curve of the backtest contained similar stretches.

The stop rules

Write these down before the first live trade. A reasonable set:

  1. Drawdown rule. If the live drawdown in R exceeds the Monte Carlo 95th percentile, stop trading the system and review. Not reduce, not "trade through it": stop.
  2. Expectancy rule. If the running average R over the last 50 (or 100 for high-frequency) trades is below zero, stop and review.
  3. Mechanism rule. If a structural change removes the reason the edge existed, stop, regardless of the numbers.
  4. Cost rule. If measured slippage exceeds twice the backtest assumption for 20 consecutive trades, stop and re-run the backtest at the measured level before continuing.
  5. Discipline rule. If you have breached the rules three times in a month, stop; the problem is not the system.

Key idea: Decide the stop conditions before going live, from the backtest's own variability. Then, when one fires, stop. Reviewing a system while continuing to trade it is how a 20R drawdown becomes a 40R one.

What "review" means

Stopping is not abandoning. Review means going back to the pipeline with the new data:

  1. Add the live trades to the data set. They are the most valuable out-of-sample data you have.
  2. Re-run the backtest with the measured slippage.
  3. Check whether the live period's conditions were represented in the original data. If not, the backtest never covered this regime.
  4. If the strategy still passes with the live period included and honest costs, and the mechanism still holds, resume at half size and monitor.
  5. If it does not pass, retire it. Write down why. The next system benefits.

A strategy that is retired after a stop rule fires has cost you a planned, bounded amount. A strategy that is tweaked while it continues to lose has cost you an unplanned one, and has also destroyed the out-of-sample value of the live data, because you have now fitted to it.

Do not tweak live

The temptation after a bad month is to add a filter that would have avoided it. This is in-sample optimisation on the most recent data, and Module 6 explained where it leads. If a change is warranted, it goes through the whole pipeline: hypothesis, backtest, out-of-sample, incubation. The live system continues unchanged until the stop rules fire or the new version has earned its place.

Situation Response
Within variance, no alarms Keep trading, keep logging
One alarm fires Stop, review per the steps above
Bad month, no alarms Keep trading; this is what the Monte Carlo predicted
Idea for an improvement Write it down; test it separately; do not change the live rules

Try it: Build the monitoring table for your system with backtest values and alarm levels in the last column. Compute the standard error band for a 50-trade average and the Monte Carlo 95th percentile drawdown. Write the five stop rules underneath with your numbers filled in. Date it.

Recap

  • Track frequency, win rate, average R, slippage, drawdown, time underwater and rule breaches against the backtest.
  • Set alarm levels from the backtest's standard error and Monte Carlo distribution, before going live.
  • Frequency and slippage changes point to a broken edge; average R within the error band points to variance.
  • When a stop rule fires, stop, then review with the live data added; resume at half size or retire.
  • Never tweak a live system in response to recent losses; improvements go through the full pipeline.

See it drawn

Original diagrams for the ideas on this page. Illustrative, not real market data.

Slippage on a market orderA buy order clears four price levels, so the average price paid is worse than the price first quoted.Buy 1,000 shares at marketpricesell orders resting (bar length = size)20.04300 shares20.03200 shares20.01200 shares20.00300 sharesnothing resting at 20.02order sweeps up the bookaverage fill 20.02SLIPPAGE0.02 a share$20.00 in totalintended 20.00Each level fills at its own price; the average is what you really paid.
Slippage on a market order. You click at 20.00, but only 300 shares are resting there, so the rest of the order fills at 20.01, 20.03 and 20.04. The average price paid is 20.02, and that two-cent gap is slippage.
An equity curve and its drawdownAn account balance rising over a year, falling from a peak to a trough, then climbing back to the old peak.ACCOUNT EQUITY$20k$12k$8k024681012TIME (MONTHS)PEAK $16,000TROUGH $12,000DRAWDOWN−25%RECOVERY
Equity curve and drawdown. An account balance plotted month by month. The fall from the $16,000 peak to the $12,000 trough is a 25% drawdown, and the shaded area lasts until the balance climbs back to the old peak.
The spread of outcomes behind an expectancyA histogram of forty trades: a tall block of small losses on the left, a low spread of larger wins on the right, and a line marking the average outcome.NUMBER OF TRADES051024 LOSSES, AVG −$20016 WINS, AVG +$600EXPECTANCY +$120−$400−$200$0+$200+$400+$600+$800PROFIT OR LOSS PER TRADEexpectancy = (40% × $600) − (60% × $200) = +$120 per trade
Expectancy: the average trade. Forty trades sorted by outcome: 24 small losses and 16 larger wins. Weighting each side by how often it happens gives the average result per trade, marked here by the dashed line at +$120.