We had the data to check our backtest, and never asked for it
For two months our backtest printed the same caveat at the bottom of every report: sampling means maximum adverse excursion is a lower bound, so the replay under-counts stop-outs and take-profit touches. It was true, we knew it was true, and we treated it as a fact of life.
It was not a fact of life. It was a join we had not written.
What the replay could see
The bot observes prices on a cron that fires every 15 minutes. Each observation is a single point: the price at that instant. The replay walks those points forward from each entry and asks, at each one, whether the stop or the take-profit has been breached.
Between two points it is blind. If a position dropped through the stop and recovered before the next observation, the replay never saw it — and scored that trade as if the stop had never been touched.
In September we started stamping each observation with the high and low of its own interval, which fixes this going forward. But it is forward-only: the diagnostic could speak for 4.6% of intervals and 19 of our 335 entries. On that sample it found one missed stop, and honestly reported that this was too thin to mean anything.
The prices were already on disk
Separately, months earlier, we had cached 15-minute OHLC bars for the same ten symbols back to January 2022 — for a different piece of work, and never connected to this one.
The cron fires on 15-minute boundaries. So each observation's forward interval maps onto exactly one cached bar: the one starting at that observation's timestamp. The high and low of that bar are precisely the prices the replay was blind to. It is a dictionary lookup.
| Lane | Coverage before | After | Entries covered |
|---|---|---|---|
| Intraday | 4.6% | 94.6% | 19/335 → 335/335 |
| Projection | 9.1% | 97.9% | 14/146 → 137/146 |
| Crypto | 0% | 0% | no cache; 5-min cadence |
What it found
At a 2.0% stop — the setting the bot ran on before we removed stops entirely — the count went from 1 missed stop to 221. Across 335 trades. That is more missed stop-touches than there are trades, so many positions breached the level in several separate intervals without the sample point ever seeing it.
| Stop width | Missed stops | Missed TPs | Deepest missed |
|---|---|---|---|
| 0.35% | 369 | 98 | −3.72% |
| 1.0% | 328 | 132 | −4.39% |
| 2.0% | 221 | 164 | −4.44% |
| 3.0% | 136 | 179 | −5.88% |
| 4.0% | 119 | 184 | −7.21% |
The deepest missed breach at a 2% stop is −4.44%: the price went more than twice the stop distance below entry, inside a single interval, and the replay scored that trade as untouched.
And it is wrong in both directions. The right-hand column is take-profit touches the replay also missed — trades the live bot would have closed at +3% and the replay held on to. Under-counting stop-outs flatters every stopped policy; under-counting take-profits changes what every policy was holding.
What we are not claiming
These are counts, not a corrected expectancy. Re-pricing the curve needs a fill convention for a stop that was touched and retraced, and an OHLC bar cannot tell you whether the stop or the take-profit came first when both were hit in the same interval. That is a real modelling decision and we have not made it, so the curve itself is untouched.
What the counts do establish is that the point-sampled curve was not a close approximation of anything. The caveat we printed for two months described a rounding error. The reality is hundreds of level touches per run.
Two lines, one conclusion
The previous post showed the stop curve was measuring market exposure rather than skill: our returns rose as stops widened, and random entries' returns rose faster. This one shows the same curve was computed on a price path that misses hundreds of level touches.
Those are independent methods reaching the same verdict, which is the strongest evidence we have produced. The first says the curve measured the wrong thing. The second says it measured it on the wrong data.