You Have Three Independent Validators. They All Miss the Same Things.

Helps identify when attestation validators are organizationally independent but epistemically correlated — the failure mode where diversity of validators does not produce diversity of judgment.

Problem

Multi-validator attestation assumes that independent validators provide independent checks. The assumption is wrong when validators share upstream dependencies that determine what they can and cannot detect.

Two validators trained on the same dataset will systematically agree — including on what they miss. Their organizational independence is real. Their epistemic independence is not. A skill that evades one validator's threat model will evade the other's with the same probability, not an independent one. The combined attestation is not stronger than either alone; it is the same check run twice under different names.

This matters because correlated validators produce a false sense of coverage. An agent operator looking at attestation badges from three validators reasonably assumes that each validator is providing an independent check. If those validators share training provenance, fine-tuning pipeline, or base model, the checks are correlated. A systematic evasion technique that works against any one of them likely works against all three — the diversification does not reduce the risk.

The organizational diversity assessment in standard attestation root analysis catches organizational overlap. It does not catch epistemic overlap across organizationally independent validators that share training lineage.

v1.1 adds a third detection path: evaluation trace correlation. When validators publish their reasoning chains (not just pass/fail verdicts), a meta-evaluator can detect correlation statistically — without requiring anyone to disclose their architecture. Two validators that consistently flag the same issues in the same order with the same reasoning structure are probably correlated, regardless of what they declare. This makes correlation observable rather than dependent on self-report.

What This Analyzes

This analyzer examines validator judgment correlation across five dimensions:

1. Training provenance disclosure — Do validators disclose the datasets, base models, or fine-tuning procedures used to develop their evaluation capabilities? Undisclosed provenance makes correlation undetectable

2. Base model overlap — Do multiple validators derive from the same foundation model? Validators that share a base model share that model's systematic biases and blind spots, regardless of organizational independence

3. Fine-tuning pipeline similarity — Were validators trained on similar security datasets or red-teaming corpora? Shared training data produces shared detection coverage — and shared detection gaps