Findings: v38 Full Portfolio Evidence

Summary

I built an adversarial multi-agent LLM framework that detects narrative-reality gaps in macro-financial markets and trades the subsequent repricing. The v38 backtest across 12 years of real market data demonstrates that the approach works: 83.6% win rate, +10.7% median P&L per trade, with out-of-sample results that hold up.

This page presents the evidence in full. Every number is backed by a reproducible backtest against yfinance historical prices.

Methodology overview

The pipeline processes 632,000+ real news headlines (Guardian, NYT, GDELT, FOMC minutes, EDGAR filings) across 6,155 indexed days from 2014 to 2026. For each detected narrative-reality gap, an 8-agent adversarial debate produces a directional verdict. Market-data-driven entry gates filter trades where the move is already priced in. A volatility-based stop-loss (3-sigma of 5-day returns) bounds downside risk.

For full architectural detail, see the Methodology page.

v38 headline results

Per-category breakdown

Commodity dominates at 100% win rate. These represent textbook Soros-style setups: structural supply/demand imbalances that mainstream narrative ignores until physical markets force repricing. The debate system excels here because the gap between “market says abundant” and “physical data says deficit” is clearly articulable.

Geopolitical at 90% reflects the system’s ability to detect sanctions, trade disruptions, and regime instability before markets fully price the second-order effects.

Currency peg trades are rare but large. The +108.9% mean is driven by EM reserve depletion trades (Turkish lira, Brazilian real, Russian ruble) where structural impossibility of maintaining the peg creates asymmetric payoff.

Central bank and real estate are harder. CB requires identifying genuine trilemma impossibility (not just disagreeing with rate guidance). RE requires an acute forcing catalyst, not just “prices are too high.” The entry gates for these categories reflect these lessons.

Out-of-sample validation

I split the data temporally: all development decisions (gate thresholds, prompt tuning, category routing) were informed by pre-2023 data. The 2023-2026 period serves as a true out-of-sample test.

The 4-point WR drop and 2.2-point median drop from in-sample to OOS is within normal statistical variation for a 42-trade sample. The system generalizes — it is not overfit to historical patterns.

Cost analysis (IBKR fee model)

I modelled realistic trading costs using Interactive Brokers fee schedules:

The 6.5-point drop from gross to net win rate reflects the impact of financing costs on longer holds. Even net of all costs, the system produces nearly +10% median per trade with a positive Sharpe.

Entry gates (v38)

The v38 entry gates are market-data-driven filters that prevent entering trades where the move is already priced in. Each gate uses observable market signals rather than hard-coded rules, making them generalizable and auditable.

These gates evolved through 38 iterations of empirical testing. Each was added only after identifying a systematic loss pattern in the backtest and validating that the gate eliminates the losses without filtering winners.

Limitations and blind spots

The system has genuine architectural limits that I do not expect further iteration to solve:

1. Single-day shocks. Events like the Swiss Franc unpeg (Jan 2015) have no narrative buildup. The system needs sustained divergence between narrative and reality.

2. Hidden leverage collapses. Archegos, SVB, and similar events involve private information about positioning that never appears in news flow until after the fact.

3. Crypto events. Crypto is underrepresented in Guardian/NYT coverage. Terra-Luna and similar events fall through the news filter.

4. Compressed events. Setups that build and resolve within 3 days do not generate enough debate signal.

5. Central bank precision. The system correctly identifies structural impossibility but sometimes enters too early. The CB gates help but do not fully solve the timing problem.

6. Financing cost on long holds. Some trades are correct directionally but the financing drag erodes net returns. The exit system mitigates this by closing trades when the gap narrows, but holds occasionally extend beyond optimal.

Evolution from v1 to v38

The pipeline went through 38 major iterations between January and June 2026:

• v1-v3: Basic detection pipeline, hand-labeled events, no exit system
• v4-v7: Feed-driven discovery, narrative collapse, signal store
• v8-v10: Adversarial exit system, intelligent instrument selection, determinism
• v11-v20: Category-specific prompt tuning, entry timing research
• v21-v30: Market-data entry gates, per-category empirical thresholds
• v31-v38: Gate refinement (CB momentum, SD own-currency, geo drawdown), OOS validation

Each iteration followed the same protocol: identify a systematic loss pattern, propose a market-data-driven gate, validate it eliminates losses without filtering winners, then run full portfolio regression.

Reproducibility

Every result on this page is reproducible:

cd development
python -u run.py --mode feed --from 2014-01-01 --to 2026-06-01 --workers 10
python -u tests/validate_lifecycle.py
python -u tests/evaluate_store.py

The signal store, backtest results, and intermediate debate transcripts are all preserved as append-only JSONL files in development/results/.

Legacy result (historical context): 204 hand-labeled events, 2022-2025

Prior to the feed-driven pipeline, Market Synthesis was evaluated on 204 historical macro-financial events (150 positive dislocations + 54 quiet periods) spanning 1971-2025, with 3 independent runs to measure consistency.

Single agent scores everything high (no discrimination) — adversarial debate achieves both detection AND precision

What worked

1. Adversarial debate produces higher-quality verdicts than single-agent.
   Multi-agent without orchestration was worse (77%). Structured debate with domain experts recovered accuracy while adding discrimination.
2. Domain routing improved precision.
   Splitting the catch-all “policy_reversal” into central_bank, geopolitical, and market_structure categories allows per-domain prompt tuning.
3. Direction-agnostic scoring + deterministic override reduced false conviction.
   The system scores WHETHER a gap exists, not which direction to trade. Per-domain thresholds force neutral when score is below domain-specific cutoff.
4. Mixed Haiku+Sonnet outperforms homogeneous Sonnet.
   Cheap advocates commit firmly, giving the expensive judge clean signal.
5. Parallelism reduced wall time 14x.
   Prosecutor, Defender, and Expert run in parallel. Trade Expression and Calibration run in parallel. Inter-event parallelism with 10 workers. 204 events in ~10 minutes.

Cross-run consistency

3 runs on the same 204 events reveal systematic vs stochastic performance:

66% of events are reliably detected — 10% are systematic blind spots that need richer context

Stability by domain

4 categories work reliably — 3 new categories need enriched context beyond headlines

Where it struggles (and why)

The 15 always-missed events fall into 4 failure modes:

These represent genuine architectural limits: the system needs narrative buildup in mainstream news to detect gaps. Single-day shocks, hidden-leverage collapses, and unpredictable policy timing remain blind spots regardless of data quality.

False positive analysis

37% of quiet periods are reliably rejected — the system's gap-detection bias triggers on any economic narrative

The 6 systematic false positives all have economic narrative framing in their context (e.g., “convergence trade,” “mid-cycle stability”). The system is biased toward finding gaps — when a quiet period has any economic story, it sometimes flags it. The deterministic no-trade override helps: when score < 50, direction is forced to neutral.

What’s next

1. Live production pilot — shadow-mode deployment against real market decisions via IBKR.
2. Options module — OTM options for asymmetric risk/reward on high-confidence gap signals.
3. Market structure category — leverage blowups remain a blind spot; exploring alternative data sources.

For pipeline architecture, see Methodology. For the exit system design, see Exit System.