Use case
Training data,
verified by Quorum.
A single scraper fooled by an anti-bot challenge can poison a training corpus with sanitized lookalike pages. Quorum, gottem's /v1/compare endpoint, fans the same URL across several vendors, SHA-256s every response, and tells you which ones agree. Majority content is ground truth; outliers are quarantined with full provenance.
Consensus, not trust
Each vendor in the comparison returns its content, its content-quality verdict, its cost, and a SHA-256 hash. Identical responses collapse into variants; the best route is chosen deterministically by quality, then cost, never by latency. You see exactly who agreed with whom.
A threshold rule that scales
The working rule from the field: keep rows where good_count >= 3 for training data, quarantine the rest with provenance instead of dropping them silently. One flag on your pipeline, and a poisoned page can't slip in on a single vendor's word.
Honest economics
Every fetch in a comparison is real and billed. The merge only dedupes the payload. Reserve Quorum for the corpora where correctness is worth multiples of a fraction of a cent, and use plain /scrape for everything else.
Start with a curl.
curl -sS "$GOTTEM_BASE_URL/v1/compare" \
-H "Authorization: Bearer $GOTTEM_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://news.ycombinator.com",
"routes": ["firecrawl.scrape", "zyte.api", "spider.smart"]
}' | jq '.best, .good_count'Questions.
- How does Quorum decide which response is the ground truth?
- Deterministically: good content quality first, then lowest catalogue cost, then route id. Latency never decides, so the pick is reproducible run over run.
- Is every route in a comparison billed?
- Yes, each fetch is real. Identical content is merged in the response payload, but the ledger shows every attempt, so the cost of certainty is always visible.