Technical Summary

Architecture

Market Synthesis uses adversarial orchestration with domain-routed experts:

Evolution from single agent to adversarial debate — orchestration is what makes multi-agent work

The current production architecture (8 agents, 4 sequential round-trips):

1. News Enrichment fetches real contemporaneous coverage (Guardian API 1992+, GNews 2004+, NYT if key available) with per-domain search strategies.
2. Classifier (Sonnet 4.5) routes event to 1 of 7 domains — this is the critical routing decision; miscategorization cascades through the entire pipeline, so it uses the strongest model.
3. Prosecutor (Haiku 4.5) argues for dislocation using domain-specific evidence (e.g., reserve depletion for FX, leverage metrics for market structure).
4. Defender (Haiku 4.5) argues against using domain-specific counter-evidence — independent from Prosecutor.
5. Domain Expert (Sonnet 4.5) provides category-specific specialist analysis.
6. Domain Adjudicator (Sonnet 4.5) synthesizes all arguments using domain-specific adjudicator prompt with containment checks — outputs verdict + confidence.
7. Gap Characterization (Haiku 4.5) describes the detected gap for fund managers (affected assets, key tension, time horizon, example actions) — runs in parallel with Calibration.
8. Calibration Agent (Haiku 4.5) adjusts probability for ECE optimization.

All LLM calls use temperature 0.0 for full determinism — the same input always produces the same output, enabling reproducible backtests and meaningful A/B comparisons.

Total: 28+ specialized prompt templates (7 Prosecutor + 7 Defender + 7 Expert + 7 Adjudicator) — each category’s entire pipeline is domain-aware, including domain-specific containment checks for sovereign debt, central bank, and geopolitical events.

Domain expert routing

Events are classified into 7 categories, each with a specialized expert prompt:

7 specialized experts — each tunable independently with domain-specific prompt engineering

Evaluation setup

• Dataset: 204 events total (150 positive + 54 quiet periods, 1971–2025)
• 7 event categories with per-category accuracy tracking
• Cross-run consistency analysis (3 runs) to separate systematic vs stochastic failures
• Intra-event parallelism: 4 round-trips instead of 8 sequential calls
• Inter-event parallelism: ThreadPoolExecutor with configurable workers
• Deterministic “no trade” override: score < 50 OR gap_rejected → direction forced to neutral

Current state (2026-05-19): v10 pipeline + adversarial exit backtest

The canonical result. Full 12.3 years (2014-01-01 → 2026-05-01) on real news feeds, with Sonnet category classifier and temperature 0.0 for full determinism.

Full statistical summary

Per-category P&L (adversarial exit backtest)

Key v10 improvements over prior versions

1. Sonnet category classifier — routing decision uses strongest model (miscategorization cascades through pipeline)
2. Temperature 0.0 everywhere — full determinism, reproducible results
3. Intelligent instrument selector — 3-layer (disk cache → static registry → Sonnet LLM fallback) with regression testing
4. RE relevance gate — filters miscategorized signals (retail/insurance/utility bankruptcies wrongly labeled as real_estate)
5. RE forcing catalyst filter — requires acute catalyst (entity failure, rate shock, contagion) for real estate signals

Calibration note

The model is slightly underconfident: it predicts ~75% probability on average when actual P&L is positive ~85% of the time. This is a benign bias for fund-manager use (less over-conviction → less over-sizing) but means the raw probability numbers are conservative.

Full stats & prior methodology: results/bigrun_20260505_full_stats.md. Latest backtest snapshot: results/adversarial_exit_20260519_064643_v10_pipeline_rerun.json.

Historical run snapshot (legacy 3-run cross-validation on 204 hand-labeled events)

Cross-run stability (positives)

Stability by category

Model assignment strategy

Sonnet for high-stakes decisions (routing, expert, judge, instrument); Haiku for committed advocacy

Key technical insights

1. Full domain specialization beats generic debate.
   All debate agents (Prosecutor, Defender, Expert, Adjudicator) now use category-specific prompts — 28+ templates total. A currency peg Prosecutor argues about reserve depletion; a sovereign debt Adjudicator has containment checks for debt scares vs actual defaults.
2. Domain-specific containment checks reduce FPR.
   Sovereign debt, central bank, and geopolitical adjudicators distinguish “scares that resolved” from “crises that happened” — two-sided rules that cap scare scores while protecting confirmed events.
3. Sonnet for routing, Haiku for advocacy.
   The category classifier uses Sonnet because miscategorization cascades through the entire pipeline. Advocates use Haiku because firm commitment produces cleaner signal for the judge.
4. Temperature 0.0 for full determinism.
   All LLM calls use temperature 0.0. Same input → same output. This enables reproducible backtests and meaningful A/B comparisons between pipeline versions.
5. Adversarial exit outperforms mechanical rules.
   An LLM weekly check (“is the gap still open?”) beats fixed hold periods + stop-losses by +3.11% per trade and reduces worst-case drawdown by 38 percentage points.
6. Intelligent instrument selection with regression testing.
   3-layer selector (disk cache → static registry → Sonnet LLM) picks the best tradeable expression. Before replacing a cached instrument, the system regression-tests against all other signals using that ticker.
7. Signal quality gates before instrument optimization.
   RE relevance gate + forcing catalyst filter remove miscategorized signals at the source — better than finding the “least bad instrument” for a wrong signal.
8. Real news enrichment with per-domain search strategies.
   Guardian API (1992+) and GNews (2004+) with category-specific search terms. Per-domain source selection (e.g., commodity events skip Guardian).
9. Deterministic guardrails prevent false conviction.
   Per-domain score/probability thresholds. When score or probability is below threshold, direction is forced to neutral regardless of LLM output.

Source files

• development/detectors/adversarial_detector.py — 8-agent detector with 28+ domain-specific prompts (including domain adjudicators)
• development/detectors/adversarial_exit_detector.py — LLM-based weekly exit check (gap still open?)
• development/shared/instrument_selector.py — 3-layer instrument selector with disk cache + regression testing
• development/shared/historical_news.py — News enrichment (Guardian/GNews/NYT APIs) with per-domain search strategies
• development/shared/domain_config.py — Per-category pipeline configuration (FRED series, tickers, thresholds, search strategies)
• development/shared/macro_data.py — Real macro data (FRED + World Bank + yfinance) with per-domain indicator selection
• development/shared/signal_extractor.py — Signal Extraction Agent (Haiku) for structured signal extraction from news
• development/shared/bedrock_client.py — AWS Bedrock client with SSO auto-refresh
• development/tests/backtest_adversarial_exit.py — Full adversarial exit backtest (303 signals, v10)
• development/tests/backtest_parallel.py — Parallel backtest with domain-specific thresholds and direction-agnostic scoring
• development/data/instrument_cache.json — Living map of signal → instrument assignments