ResearchStatistics
Deflated Sharpe ratio and the probability of backtest overfitting
Selecting a strategy from many trials changes how its Sharpe should be interpreted. Deflated Sharpe and the probability of backtest overfitting account for different parts of that selection process.
Editorial note: this article was prepared with AI assistance and checked against the product and methodology pages listed below. Illustrative examples explain the methods. They do not describe investment results.
- the deflated-Sharpe gate as it ships: The 0.90 threshold and the inputs it is computed from are the ones in the grading harness.
How selection changes a Sharpe estimate
Searching across candidate strategies changes the meaning of the selected Sharpe. Sampling noise can produce a strong-looking winner even when the underlying strategies have no persistent advantage. Retain the candidate set and search procedure when reporting the selected result.
Selecting a candidate is a statistical operation. Report the selected Sharpe together with the search that produced it, so readers can assess the effect of selection.
E[max of N standard normals] ≈ √(2 ln N)Step one: the Probabilistic Sharpe Ratio
Before dealing with selection, deal with the fact that a Sharpe ratio is itself estimated from a finite, non-normal sample. The Probabilistic Sharpe Ratio (PSR) answers: given this track record's length, skew and kurtosis, what is the probability that the true Sharpe exceeds some benchmark?
PSR(S*) = Φ( (Ŝ − S*) · √(n−1) / √(1 − γ₃Ŝ + ((γ₄−1)/4)Ŝ²) )A short sample leaves more uncertainty around its estimated Sharpe. Skew and kurtosis also affect the estimate's standard error. Report those inputs beside PSR so a reviewer can see the assumptions behind its probability.
Step two: the Deflated Sharpe Ratio
The Deflated Sharpe Ratio (DSR) is PSR with the benchmark set to the Sharpe you would expect from the best of N trials on strategies with no edge. Instead of asking “is this Sharpe above zero?”, it asks “is this Sharpe above what the search alone would have produced?”
S* = σ(Ŝ) · [ (1 − γ)Φ⁻¹(1 − 1/N) + γΦ⁻¹(1 − 1/(N·e)) ]The result depends on the effective trial count N and the dispersion σ(Ŝ) across trials. Missing variants can understate either input. Keep failed variants and their Sharpe estimates alongside selected results, and state how dependent trials were handled.
- Return series
- Keep the sampling frequency and cost assumptions fixed
- Trial history
- Retain failed variants and their Sharpe estimates
- Selection benchmark
- Estimate it using the trial dispersion and effective count
- Verdict
- Read the statistic with the published gate threshold
TradePolaris requires a Deflated Sharpe above 0.90 for a run to pass, computed on the per-trade return series with the trial count the run actually used. A run can have a good Sharpe and fail here, and that is the gate doing its job.
The other question: probability of backtest overfitting
DSR evaluates an observed Sharpe against a selection-adjusted benchmark. PBO evaluates the selection procedure across train and test splits. It measures how often the configuration selected in training ranks below the median in testing under the stated procedure.
- Arrange the return matrix with trials on one axis and time on the other. Divide the time axis into an even number of equal blocks.
- Enumerate balanced combinations of training and testing blocks. The blocks remain intact, while a combination can span noncontiguous parts of the history.
- For each split, find the trial that ranked best in training, then record where that same trial ranked in testing.
- PBO is the fraction of splits in which the in-sample winner landed in the bottom half out of sample.
PBO estimates how often the training winner falls below the median in held-out blocks under the chosen procedure. Interpret it with the candidate set, block construction and sample length. It is not a forecast of the chance of losing money.
| Deflated Sharpe | PBO | What it means |
|---|---|---|
| High | Low | The reported result clears its benchmark and the selection procedure has fewer below-median held-out ranks. Check the assumptions. |
| High | High | One result clears its benchmark while the selection procedure shows frequent below-median held-out ranks. |
| Low | Low | The reported result has limited evidence against its benchmark, even when the selection procedure has fewer rank failures. |
| Low | High | Both measures raise questions about the reported result and the search procedure. |
What a reviewable report contains
- The trial count, including abandoned variants, with the counting method stated.
- The dispersion of Sharpe ratios across trials, alongside the selected trial's Sharpe.
- Skew and excess kurtosis of the return series, because both move PSR materially.
- The Deflated Sharpe, with the benchmark S* it was deflated against.
- PBO, with the number of blocks used.
- Whether the underlying series is fitted or held out. A selection correction does not replace an independent evaluation of the chosen strategy. See walk-forward testing vs backtesting.
References
- Bailey & López de Prado, The Deflated Sharpe Ratio (2014): Source of the S* benchmark and the deflation itself.
- Bailey & López de Prado, The Sharpe Ratio Efficient Frontier (2012): The Probabilistic Sharpe Ratio and its skew/kurtosis correction.
- Bailey, Borwein, López de Prado & Zhu, The Probability of Backtest Overfitting (2015): The combinatorially symmetric cross-validation procedure behind PBO.
- Harvey, Liu & Zhu, …and the Cross-Section of Expected Returns (2016): Why multiple-testing corrections belong in finance at all.
Price the search into the result.
TradePolaris deflates every run against the trials it actually took and reports the verdict beside the raw number.
Read the gate methodology →TradePolaris provides research software for paper trading. It does not provide investment advice. Backtested and hypothetical results do not predict future performance. Trading is paper (simulated) only. See our Disclaimer.