What Monte Carlo can and cannot tell you about a backtest
Shuffling a strategy's trades shows how deep its drawdown could have been. It cannot see trades that depend on each other, or a market it never met.
A backtest, or a signal history, is one path. The trades happened in one order, and the drawdown on the report is the deepest fall of that one order. Had the same trades come in a different order, the drawdown would have been different. A Monte Carlo simulation asks how different.
It does this by brute force. Take the list of trade results. Shuffle it into a random order, walk the balance through it, and note the worst drawdown. Do that hundreds or thousands of times. The spread of those drawdowns is the answer: how deep a fall these trades can produce, and how unusual the real one was.
Two ways to shuffle
There are two common methods, and they answer different questions.
Reorder uses each trade exactly once, in a new order. The final balance is always the same, because the same trades add up to the same total. Only the path changes. It answers: how deep could the drawdown have been, with these exact trades?
Bootstrap draws trades from the list at random, with replacement, so a trade can appear twice or not at all. The final balance now changes too. It answers: what might another run of the same length look like, if the future trades resemble these?
Try both below. The solid blue line is the real order. The fan around it holds the middle 90% of the simulated runs.
- The real order
- Median run
- Best 5%
- Worst 5%
- Worst drawdown, in the real order
- 38%
- Median worst drawdown, shuffled
- 33%
- 1 run in 20 was worse than
- 53%
- Runs deeper than the real drawdown
- 29%
What it can tell you
How lucky the real order was. Signal A made 520 trades. Its real drawdown was 38%. Shuffled 500 times, the median drawdown was 33%, and 1 run in 20 fell 53% or more. The real path was ordinary: 29% of the shuffles were worse. Plan for the bad end of the spread, not for the one path you saw.
How thin a good record can be. Signal B grew its start balance of 1,000 by 23,434. Its real drawdown was 28%. But a handful of large losses sit among its 552 trades. Put them together early, before the profits had piled up, and the balance does not survive: in 1 shuffled order out of 20, the drawdown reached 100%. The same trades, in a less friendly order, end the account.
- signal B: real drawdown
- 28%
- median of the shuffled runs
- 21%
- 1 run in 20 was this deep or worse
- 100%
How uncertain the profit is. Signal A ended 4 up on a start of 150. Bootstrap it, and the middle 90% of runs end between 58 and 255. The profit on the report is well inside the noise. A strategy whose result could as easily have been a loss has not shown an edge yet.
What it cannot tell you
Trades that depend on each other. A shuffle treats every trade as a separate coin toss. A grid is not built that way. It opens a chain of positions as the price moves against it, and when the chain loses, its trades lose together. Shuffling scatters that chain across the history, so the simulated drawdowns come out too mild.
Signal C is a grid that went 7 levels deep. Its real drawdown was 34%. The median shuffle gave 22%, and only 11% of the shuffles fell further than the real path did. That does not mean the real path was unlucky. It means the losses really did arrive together, and the shuffle cannot know that.
Anything that is not in the list. A simulation can only rearrange the trades it was given. It cannot see a market the strategy has not met: a flash crash, a central bank surprise, a regime that lasts longer than the backtest. It does not see open losses while positions were held, only closed results. And it does not see the trades a broker would have filled at a worse price, so the spread and slippage of your own account need a separate test.
Deposits and withdrawals. The simulation walks the trade results from the first balance. A signal topped up after each loss looks safer on its balance curve than its trades were. Monetio runs the same simulation on the trade results alone for this reason; see how deposits hide drawdown.
What to do with your own report
- Compare the real drawdown with the shuffled ones. If 1 run in 20 is much deeper than the real drawdown, size your account for that run, not for the report.
- Bootstrap the profit. If the middle of the runs includes a loss, the record has not shown an edge yet.
- If the strategy averages down or uses a grid, treat the Monte Carlo drawdown as a minimum. Look at the grid depth instead.
Monetio runs both methods on every report and shows the drawdown percentiles, the chance of a deeper drawdown than the real one, and the bootstrap range of the final profit.