ForwardCheck-AI: Building a 6-Agent Adversarial Fact-Checking Newsroom

Most AI fact-checkers work like a single journalist with a search engine. They retrieve evidence, compare it to a claim, and produce a verdict. This works for simple cases — "Did X say Y?" — but falls apart on the claims that actually matter: nuanced, multi-source, politically charged statements where the truth isn't sitting in a single Wikipedia paragraph.

We wanted something closer to how a real newsroom operates. Not one reporter, but a team. Investigators who dig independently. An editor who challenges their conclusions. A final arbiter who weighs conflicting evidence and makes the call.

ForwardCheck-AI is that system: six specialized agents running on Telegram, collaborating adversarially to produce fact-check verdicts that hold up under scrutiny.

The architecture: adversarial by design

The core insight driving ForwardCheck-AI is that validation through agreement is weaker than validation through attempted refutation. If three agents independently confirm a claim, that's useful. But if an agent specifically tasked with disproving the claim fails to find counter-evidence, that's a much stronger signal.

The system runs six agents in a structured pipeline:

Intake Agent — Receives forwarded messages on Telegram, extracts claims, deduplicates against recent checks, and routes to the investigation team.

Investigator Agents (3x) — Three parallel investigators each pursue different evidence strategies. One focuses on primary sources and official records. One searches news archives and journalistic coverage. One checks social media provenance and temporal patterns. They work independently — no shared state during investigation.

Devil's Advocate — This is where the architecture gets interesting. The Devil's Advocate receives all three investigation reports and is prompted with a single directive: find reasons the conclusion is wrong. It doesn't try to be balanced. It actively looks for counter-evidence, methodological weaknesses, and alternative explanations. Its job is to stress-test, not to agree.

Judge — The Judge receives the original investigations and the Devil's Advocate's challenges. It synthesizes a final verdict with confidence scores, citing specific evidence for and against.

interface FactCheckPipeline {
  intake: (message: TelegramMessage) => Claim[];
  investigate: (claim: Claim) => Promise<Investigation[]>;  // 3 parallel
  challenge: (investigations: Investigation[]) => Challenge;
  judge: (investigations: Investigation[], challenge: Challenge) => Verdict;
}

Why adversarial beats consensus

Early prototypes used a simpler architecture: three investigators plus a majority-vote synthesizer. The results looked good on surface metrics but had a systematic failure mode — confirmation cascades.

When all three investigators found evidence supporting the same conclusion, the synthesizer would produce a high-confidence verdict. But in roughly 15% of cases, all three had found the same misleading evidence from the same original source, propagated across multiple outlets. Three agents agreeing didn't mean three independent validations. It meant one piece of evidence counted three times.

The Devil's Advocate breaks this pattern. It's structurally incentivized to find the gap. In the confirmation cascade scenario, it would flag: "All three investigations cite derivatives of the same Reuters wire report. No independent primary source confirms the core claim."

The key metric shifted from agreement rate to challenge survival rate — how often does the original conclusion hold up after adversarial scrutiny?

Orchestration with Beads

Six agents across multiple context windows need coordination. We use Beads, a graph-based task tracker built for AI agent workflows, to manage the pipeline:

bd create "Fact-check: [claim summary]" --type task
bd create "Investigate: primary sources" --parent $CLAIM_ID --deps ""
bd create "Investigate: news archives" --parent $CLAIM_ID --deps ""
bd create "Investigate: social provenance" --parent $CLAIM_ID --deps ""
bd create "Devil's Advocate challenge" --parent $CLAIM_ID --deps "$INV1,$INV2,$INV3"
bd create "Judge synthesis" --parent $CLAIM_ID --deps "$CHALLENGE_ID"

Each investigation runs as an independent task. The Devil's Advocate task is blocked until all three investigations complete. The Judge is blocked until the challenge completes. Beads enforces this dependency graph — no agent runs before its inputs are ready.

This matters more than it sounds. When agents run on separate context windows with potential resets, you need a source of truth for "what has been done" and "what can run next" that lives outside any single agent's memory. Beads is that source of truth.

What the Devil's Advocate actually does

The Devil's Advocate prompt is deliberately one-sided. It's not asked to be fair. It's asked to break things:

You are a Devil's Advocate in a fact-checking pipeline. Your role is
adversarial: find every reason the investigators' conclusion might be wrong.

You will receive 3 independent investigation reports about a claim.
Your job:
1. Identify evidence gaps — what WASN'T investigated?
2. Find counter-evidence the investigators missed
3. Challenge methodology — are sources independent or derivative?
4. Propose alternative explanations for the same evidence
5. Rate the robustness of each investigation (1-5)

Do NOT attempt balance. Do NOT agree with the investigators unless you
genuinely cannot find weaknesses. Your value comes from finding problems,
not from being agreeable.

This prompt design reflects a broader principle: agent specialization through asymmetric objectives. The investigators optimize for evidence coverage. The Devil's Advocate optimizes for evidence destruction. Neither alone produces reliable output. Together, they produce verdicts that have survived genuine adversarial pressure.

Lessons from production

After running ForwardCheck-AI on several hundred claims, three patterns emerged:

The Devil's Advocate is most valuable when it fails. A challenge that finds nothing wrong is a stronger endorsement than three agreeing investigators. We track "clean challenge" rate as a quality signal — claims where the Devil's Advocate couldn't find meaningful weaknesses.

Parallel investigation with different strategies matters more than parallel investigation with the same strategy. Early versions used three identical investigator prompts. Results overlapped ~70%. Specializing each investigator (primary sources, news, social media) reduced overlap to ~30% and caught more claims where evidence existed in only one domain.

The Judge needs the challenge, not just the investigations. We tested a variant where the Judge only saw the three investigations (no Devil's Advocate input). Verdict quality dropped measurably. The Judge performs better when it can see which specific weaknesses were raised and whether they were addressable — it's making a meta-judgment about the investigation's robustness, not just the evidence.

The Anthropic Hackathon connection

ForwardCheck-AI was built for the Anthropic "Built with Opus" Hackathon (selected in the top 500 of 13,000 submissions). The constraint of the hackathon forced a useful discipline: build something that works end-to-end in a week, not a demo that works in a notebook.

The Telegram interface was a deliberate choice. It forced the system to handle real user input — forwarded messages with missing context, screenshots of text, claims in multiple languages. If your fact-checker only works on clean, English-language text claims, it doesn't work.

What this pattern generalizes to

The adversarial multi-agent pattern isn't specific to fact-checking. Any domain where false confidence is more dangerous than uncertainty benefits from this architecture:

• Code review — generators write code, an adversarial reviewer tries to break it, a judge decides if the code ships
• Medical triage — diagnostic agents propose explanations, a challenger looks for contradicting symptoms, a synthesizer produces the differential
• Investment analysis — bull and bear analysts make independent cases, a portfolio manager weighs the arguments

The common thread: replace single-agent confidence with multi-agent earned confidence. Make the system prove its conclusions survived scrutiny, not just that it produced them.