Research

When models agree, what evidence says they are right?

Agreement is a signal, not ground truth. Our research asks when independent evidence should confirm, qualify or overturn it.

Test a Recherche Judge trace.

Toggle evidence categories to see why the verdict does not depend on how many models agree.

Claim

“Processing remains exclusively in the EEA.”

Recherche Judge

NOT ESTABLISHED

Available evidence remains insufficient to establish exclusive EEA processing.

PrincipleConsensus does not decide.The verdict follows evidence, authority, freshness and provenance.
Recherche Judge

How the Judge adjudicates a claim.

The number of models in agreement does not decide the verdict. Model positions are tested against the available evidence.

Claim extracted from AI responses“Processing remains exclusively in the EEA.”
OpenAIOpenAISUPPORTS
ClaudeClaudeUNCERTAIN
GeminiGeminiSUPPORTS
MistralMistralUNCERTAIN
xAIGrokCONTRADICTS
DeepSeekDeepSeekUNCERTAIN
QwenQwenSUPPORTS
LlamaLlamaCONTRADICTS
Contract / DPA
Regional documentation
Subprocessor register
Policies / documentation

Recherche Judge

PROPRIETARY MODEL
EvidenceAuthorityFreshnessContradictionsUncertaintyProvenance
SUPPORTED

The evidence establishes the claim.

NOT ESTABLISHED

Model consensus is not enough: the available evidence does not establish exclusive EEA processing.

CONTRADICTED

The available evidence contradicts the claim.

Six falsifiable research questions.

The programme is organised around measurable questions whose answers can change as models, sources and deployment conditions evolve.

01

Correlated error

Measure when different providers fail together and whether additional models add genuinely independent information.

02

Claim-level adjudication

Evaluate whether verdicts based on evidence outperform whole-answer voting or generic LLM judging.

03

Evidence quality

Model source authority, freshness, provenance and contradictory evidence rather than treating every citation equally.

04

Calibration

Measure whether confidence and unresolved states correspond to observed correctness across domains and time.

05

Verification economics

Study when deeper verification improves quality enough to justify added cost and latency.

06

Regret de routage

Estimate whether another model or verification strategy could have achieved comparable quality at lower cost or latency.

How we know whether the system is improving

Benchmarks must be versioned because models, providers and sources change. Results should distinguish observed measurements from targets.

01
Versioned evaluation datasets
02
Claim-level factual accuracy and critical-error rate
03
Judge calibration and reviewer precision
04
Cost, P50/P95 latency and verification depth
05
Reproducibility across model and source versions

Failure modes are part of the research result.

Model consensus can be correlated.Different companies do not guarantee independent reasoning.
Judges can be biased.A persuasive answer can influence an evaluator unless evidence and calibration constrain it.
Sources change.Freshness and provenance matter when a claim depends on current documentation or registers.
Verification is not free.More checking adds cost and latency; the optimal depth depends on consequence and uncertainty.
Benchmarks age.Model releases can invalidate previous rankings and require repeated evaluation.
Uncertainty must survive synthesis.The final answer should preserve unresolved points instead of forcing false certainty.

Current maturity

Recherche.fr is a research prototype built on an existing multi-provider platform. Core orchestration and comparison foundations exist; claim-level adjudication, Judge calibration, adaptive verification and representative industrial validation remain active R&D.

Interactive product model

A research question must be allowed to fail.

Explore hypothesis, measurement and falsification without turning a conceptual schematic into a fake benchmark.

Illustrative · local state only
Research question
View
Hypothesis
Measure
Falsification criterion
Calibration state

No numeric value here is presented as observed performance.

Evaluation protocol

Recherche-Eval-01 — protocol before performance claims.

The first public benchmark should answer one narrow question: does claim-level adjudication against independent evidence reduce false establishment compared with simpler baselines? No performance result is published here until the benchmark, ground truth and evaluator protocol are frozen.

Research questionEvidence-aware adjudication vs simpler baselines

Measure whether the system reduces claim-level factual errors and false “established” verdicts without hiding uncertainty.

BaselinesSingle strong model · majority vote · generic LLM-as-a-Judge

Recherche.fr should be compared against approaches that are cheaper and easier to deploy, not against a straw man.

Primary measuresClaim accuracy · false-establishment rate · critical-error rate · abstention/calibration

Cost and P50/P95 latency are tracked alongside quality because verification depth has an operational price.

Adversarial slicesPersuasive wrong answer · correlated error · stale evidence · poisoned/low-authority evidence

These cases test whether the Judge follows evidence rather than confidence, style, majority or source presentation.

Dataset statusVersioned benchmark under construction

Claims require controlled ground truth, source provenance and a frozen evaluation policy before scores are meaningful.

Release ruleNo public performance claim before protocol freeze

Measured results will be separated from targets and illustrative examples.

Current status: evaluation design. Recherche-Eval-01 is a protocol target, not a published benchmark result.
Adversarial evaluation

Persuasive wrong answer test.

A system that only handles obvious hallucinations is not enough. One of the decisive evaluation slices is a false answer that is fluent, confident, well structured and supported by a weak but plausible-looking source.

SetupSeveral models repeat the same wrong claim

The agreement is intentionally correlated and the strongest response is written persuasively.

Evidence trapA low-authority or duplicated source appears to support it

The verification layer must distinguish independent evidence from repetition or source-quality theatre.

Pass criterionDo not mark the claim Established without adequate independent evidence

A correct outcome may be Not established, Contradicted or explicitly unresolved depending on the evidence.

Bias checksBlind provider identity · vary answer order · vary length/style

The verdict should not change simply because the wrong answer is longer, more polished or shown first.

No result is claimed yet. This is a required evaluation case for Judge calibration and robustness, not a demonstration of current performance.