The Magnitude Paradox: A 99.8% Accurate Detector We Can't Trade

We built a detector that identifies market instability with 99.8% accuracy. We have no idea how to trade it.

That's not a hook. That's not false modesty designed to reel you in before a pitch. That is the literal state of our research after six months, 13,000+ configurations, two instruments, five timeframes, and every signal processing technique we could throw at the problem.

We can tell you when the market is about to do something big. We cannot tell you which direction. And we have the data to prove that the direction genuinely does not exist in this signal — not that we haven't found it yet, but that it isn't there.

This is the story of building something that works almost perfectly at the thing nobody asks about, and fails completely at the thing everyone assumes matters most.

What We Built

The core idea is deceptively simple: markets operate on multiple timeframes simultaneously. A scalper on the 1-minute chart, a swing trader on the 4-hour, an institutional player on the daily — they're all looking at the same instrument but seeing different pictures. Most of the time, those pictures roughly agree. Price trends up on the 5-minute, the 15-minute confirms, the hourly agrees. Coherence.

Sometimes they don't agree. The 5-minute says up, the 15-minute says sideways, the hourly says down. Incoherence. Disagreement across timeframes. And when that disagreement reaches a critical threshold, something is about to break.

We formalized this intuition into what we call a pyramidal multi-timeframe coherence detector. Here's the architecture:

Hidden Markov Models (HMMs) sit at the base. Each timeframe gets its own HMM that classifies the current market regime — trending, mean-reverting, volatile, quiet. These aren't simple moving average crossovers or RSI thresholds. HMMs model the unobservable state of the market as a probabilistic system, inferring regime from observable price action using the forward algorithm. Critically, our inference is causal — each HMM state estimate uses only past and present data. No future bars. No lookahead.

The pyramidal structure layers these HMMs across multiple timeframes — 5-minute, 15-minute, 30-minute, 1-hour, and 4-hour — and monitors the agreement between their regime classifications. When the 5-minute HMM says "trending" and the 1-hour HMM says "mean-reverting," that's a coherence breakdown. When all timeframes disagree with each other simultaneously, you're looking at systemic instability.

The coherence score is a normalized measure of cross-timeframe agreement. High coherence means the market is behaving consistently across scales. Low coherence means the fractal structure of the market has broken down — different participants on different horizons are seeing fundamentally different things. That's when large moves happen.

We tested this across 6.7 years of MES and MNQ 5-minute data — that's Micro E-mini S&P 500 and Micro E-mini Nasdaq-100 futures. Over 470,000 bars of data. The results:

✓99.8% true positive rate for detecting instability events that preceded significant price moves
✓Instability events reliably preceded moves of 2+ standard deviations within the subsequent 1-4 bars
✓The detector identified regime transitions an average of 2-3 bars before the large move materialized
✓False positive rate held below 0.4% across both instruments and the full sample period

These are not backtested-until-they-work numbers. The HMM parameters were estimated on a training window and tested out-of-sample. The coherence thresholds were fixed, not optimized per-instrument. The 99.8% held across both MES and MNQ independently.

We had built, by any reasonable measure, an exceptional detector of market instability.

Then we tried to trade it.

The Paradox

When we broke down the instability events by the direction of the subsequent move, the distribution was approximately 52% in one direction, 48% in the other. Essentially a coin flip. And that 52% wasn't even stable — it wobbled between 49% and 54% depending on the subsample, the instrument, and the lookback window.

We initially assumed this was a data issue. Or a parameter issue. Or a timeframe issue. Surely if the detector knew when something big was coming, there had to be directional information buried somewhere in the signal. The magnitude was so clean — there had to be an edge in direction too.

There wasn't.

We didn't arrive at this conclusion casually. We arrived at it after systematically testing 13,000+ configurations across every reasonable (and several unreasonable) approaches to extracting directional information from the coherence signal.

What We Tried

Phase 1: Directional Gating — 726 Configurations

The first approach was straightforward: use the coherence breakdown as a filter and combine it with directional indicators. We tested 726 configurations combining the coherence signal with momentum indicators, volume-weighted directional metrics, microstructure imbalance measures, cross-instrument relative strength, and volatility skew approximations.

Result: No configuration produced directional accuracy statistically distinguishable from 50%.

The best performer hit 54.1% — well within random variation given our sample size.

Phase 2: Magnitude Exploitation — 8,520 Configurations

If direction is random but magnitude is predictable, maybe we can profit from magnitude alone. This is the volatility trader's thesis: who cares which way it goes if you know how far it goes?

We tested 8,520 configurations exploring magnitude-based position sizing, straddle-equivalent synthetic positions, breakout capture with symmetric entry, adaptive stop placement, and time-based exits.

Result: Magnitude alone could not overcome transaction costs in any configuration.

You're entering two positions, one wins and one loses, and the spread + commissions + slippage eat the difference.

Phase 3: Timeframe Shotgun — 3,500 Configurations Across 5 Timeframes

Maybe the direction wasn't in the aggregate coherence signal but in which specific timeframe broke first. We tested 3,500 configurations examining lead-lag analysis, breakdown sequences, speed of breakdown, partial coherence scores, and timeframe-specific transition probabilities.

Result: Zero incremental information about direction.

The lead-lag structure was real and interesting — fast timeframes did break first — but the direction of the subsequent move was independent of which timeframe led.

Phase 4: Hilbert Transform and DSP Phase Analysis

This was our most sophisticated attempt, and the one that taught us the most about methodological rigor.

The Hilbert transform converts a real-valued signal into an analytic signal with both amplitude and phase components. We implemented Hilbert-based phase analysis and initially got exciting results. Directional accuracy jumped to 61% in some configurations.

We hadn't cracked it. We'd introduced lookahead bias.

The standard scipy implementation of the Hilbert transform uses the Fast Fourier Transform (FFT) under the hood. FFT, by definition, operates on the entire signal — past, present, and future data points. When you compute the Hilbert transform at bar 1000, the FFT is secretly using information from bars 1001, 1002, ... all the way to the end of your dataset.

⚠️ The Lesson

When we built a causal rolling-window alternative using only past data, directional accuracy dropped back to 51.3%. Coin flip. The "directional edge" had been entirely an artifact of lookahead bias.

This is exactly the kind of error that produces impressive backtests and catastrophic live results.

What This Means

After 13,000+ configurations, here is what we can state with high confidence:

The coherence breakdown signal contains genuine, exploitable information about the magnitude of upcoming price moves.

This is not curve-fit. It holds out-of-sample, across instruments, across timeframes, and across years.

The coherence breakdown signal contains zero information about the direction of upcoming price moves.

This is not "we haven't found it yet." Direction during instability events is genuinely random — approximately 52/48 with no stable skew.

The paradox, stated plainly: we have a near-perfect detector of when the market will make a big move, and that information alone is not sufficient to generate positive expectancy net of transaction costs through directional trading.

The Open Question

So here we are. We have a signal that most quant teams would kill for — a 99.8% accurate real-time detector of impending market instability — and we genuinely don't know the optimal way to monetize it.

We're actively exploring several approaches:

Options straddles. The textbook answer to "I know volatility is coming but not direction." The problem: options on MES/MNQ have wide spreads, timing must be precise, and liquidity isn't always there.

VIX and volatility products. If instability is coming, go long volatility. The problem: VIX derivatives have complex term structure, and the timing mismatch between our signal (minutes to hours) and VIX product granularity is non-trivial.

Defensive filter. Instead of trading into instability, trade away from it. Use the coherence signal to flatten positions before large moves. This might be the highest-value application — not a standalone strategy but a risk overlay.

Mean-reversion fade. Large moves triggered by instability events may overshoot. A mean-reversion entry after the move completes might capture the reversion.

Spread and pairs trading. If MES and MNQ both exhibit instability simultaneously but move in different directions, the spread between them might be tradable.

Auction theory integration. Instability events likely correspond to transitions between balance areas. Integrating our signal with volume profile analysis might provide the directional context we can't extract from price alone.

What Would You Do With This Signal?

This isn't a rhetorical question. We're asking.

If someone handed you a real-time signal that said "within the next 5-20 minutes, this instrument will move 2+ standard deviations from its current price, but I can't tell you which direction" — what would you do with it?

Some specific questions we're wrestling with:

Is there a futures-only way to monetize non-directional magnitude prediction?
Has anyone successfully used instability detection as a pure risk filter?
Are there volatility products with sufficient granularity to capture event-level volatility spikes?
Is the mean-reversion-after-instability thesis well-supported in the literature?

Why We're Sharing This

First, the detection methodology isn't the moat. HMMs are well-understood. Multi-timeframe analysis is well-understood. Anyone with the skills to implement this from a blog post already has the skills to build it independently.

Second, the hard problem — monetization — is unsolved, and we think collaborative exploration will get to an answer faster than isolated research.

Third, radical transparency builds trust. No black box. No mystery sauce. Just a well-characterized signal and a documented journey.

The Bottom Line

We built something that works. It detects market instability — real, structural, multi-timeframe coherence breakdowns — with 99.8% accuracy across 6.7 years and 470,000+ bars of futures data. It uses causal HMM inference with no lookahead bias. It works on both MES and MNQ. It's real.

And we can't trade it. Yet.

The magnitude is there. The direction isn't. 13,000+ configurations confirm it. The Hilbert transform teased us with a 61% directional edge that turned out to be lookahead bias from scipy's FFT — and catching that error might be the most valuable thing we've done in this entire project.

We're not stuck. We're at the interesting part. The part where a well-characterized signal meets an open problem.

So: what would you do with this signal?

Drop your thoughts below. DM us. Quote-tweet with your take. We're reading everything.