Our replay walked a price path sampled every 15 minutes and apologised for it in every report. The missing prices had been cached on disk since 2022. Once joined, the replay turned out to have missed 221 stop-outs it never charged.
For two months a rising curve told us wider stops were better, and we acted on it. Running the same curve with random entries showed it rising faster — the slope was time in a rising market, and our entries captured less of it than chance did.
One entry slot out of 23 beat a random-entry control with p = 0.020. Correcting for the fact that we had searched all 23 put it at p = 0.467 — almost exactly the average result of searching noise.
Inside a strategy with provably zero edge, one 15-minute window beat a random-entry control at the 5% level. Removing the single year our parameters were fitted on took it below the threshold.
Two months of live paper trading said the strategy worked. One out-of-sample test said it was worse than picking entries at random. We retired it the same day.
The thesis was that a tight stop was converting temporary drawdowns into permanent ones. It was right. Fixing it still did not produce an edge — and that distinction took us two more months to understand.