citation-check-skill

name citation-check-skill description Vision-enabled verification gate with web search. Use when users want to (1) verify slides/reports/PDFs/images against authoritative online sources, (2) validate that citations actually exist and say what's claimed, (3) check charts/graphs/tables for accuracy, (4) audit AI-generated content in doc-only mode (no external knowledge). Two modes - search mode validates against web, doc-only mode ensures everything traces to provided documents. Supports content in any language. Citation & Hallucination Checker v2 Verification tool with vision + web search. Validates every claim against authoritative sources or provided documents. Works with content in any language. Design principle: Deterministic verification. Same input → Same output. Two Verification Modes Mode 1: Search Verification (Default) Searches web for authoritative sources Validates citations actually exist Checks if cited sources say what's claimed Finds original data for statistics Mode 2: Doc-Only Verification User provides source document(s) EVERYTHING must trace to those docs Flags anything that appears to come from external knowledge Trigger: "only use this document" / "verify against the PDF only" / "don't search the web" Two-Pass Architecture Critical: Always use two separate passes. Never interleave extraction and verification. Pass 1: Extraction Only Read entire document/slides/images Extract ALL claims using the Claim Extraction Rules below Output numbered list: [claim_id] | [claim_text] | [claim_type] | [location] NO verification in this pass Present extraction to user for confirmation before proceeding Pass 2: Verification Only Take Pass 1 output as fixed input Verify each claim_id in sequential order NO re-extraction allowed — work only with Pass 1 claims Apply Status Decision Tree to each claim Generate final report This prevents "discovering new claims" mid-verification and ensures consistency. Claim Extraction Rules (Exhaustive) Extract ONLY these claim types. Apply rules strictly — no judgment calls. EXTRACT as claims: Type Pattern Example Statistic Any number with unit/context (%, $, count, ratio, decimal) "92.3% accuracy", "$4.7B market" Comparative X is [comparative] than Y "3x faster than baseline" Temporal Time-bound assertion "In 2024, adoption reached..." Attribution Claim tied to source "According to WHO...", "Smith et al. found..." Causal X causes/leads to/results in Y "This reduces latency by..." Existence Asserts something exists/is true "There are 500M users", "The model supports..." Ranking Position claims "largest", "first", "top 3" Quote Direct quotation Any text in quotation marks attributed to source DO NOT extract as claims: Type Example Reason Definitions "Machine learning is a subset of AI" Definitional, not factual claim Opinions marked as such "We believe...", "In our view..." Explicitly subjective Hypotheticals "If adoption continues...", "Could potentially..." Speculative Questions "What drives growth?" Not an assertion Future predictions without source "Will reach $10B by 2030" Unless citing a forecast report Methodology descriptions "We used PyTorch 2.0" Process, not factual claim Acknowledgments "Thanks to our collaborators" Not verifiable Extraction Output Format [C01] | "Model achieves 96.555% accuracy on ImageNet" | Statistic | Slide 3, bullet 2 [C02] | "Outperforms GPT-4 by 12% on reasoning tasks" | Comparative | Slide 3, bullet 3 [C03] | "According to Chen et al. (2024), transformers scale linearly" | Attribution | Slide 5, para 1 [C04] | "Market size reached $4.7B in 2024" | Statistic + Temporal | Slide 7, chart title Status Decision Tree Apply this tree to EVERY claim. Follow exactly — no shortcuts. START │ ├─ Is this a CITATION claim (references a paper/report/source)? │ ├─ YES → Go to CITATION VALIDATION │ └─ NO → Go to STATISTIC/FACT VALIDATION │ │ CITATION VALIDATION │ ├─ Step 1: Does the cited source exist? │ │ Run ALL mandatory search queries (see Search Templates) │ │ │ ├─ NO → Status: "Citation Not Found" │ │ Issue: "Cannot locate [citation] in any database" │ │ STOP │ │ │ └─ YES → Step 2: Does source contain the claimed topic? │ │ │ ├─ NO → Status: "Misquoted" │ │ Issue: "Source exists but does not discuss [topic]" │ │ STOP │ │ │ └─ YES → Step 3: Does source support the exact claim? │ │ │ ├─ YES (exact match) → Status: "Verified" │ │ Confidence: "exact" │ │ │ ├─ YES (paraphrase, same meaning) → Status: "Verified" │ │ Confidence: "paraphrase" │ │ │ ├─ PARTIALLY (missing context) → Status: "Misleading" │ │ Issue: "Claim omits critical context: [what's missing]" │ │ │ └─ NO (contradicts) → Status: "Hallucination" │ Issue: "Source says [X], claim says [Y]" │ │ STATISTIC/FACT VALIDATION │ ├─ Step 1: Can you find an authoritative source? │ │ Run ALL mandatory search queries (see Search Templates) │ │ │ ├─ NO (no source found) → Status: "Unverified" │ │ Issue: "No authoritative source found" │ │ STOP │ │ │ └─ YES → Step 2: Do values match EXACTLY? │ │ │ ├─ YES → Status: "Verified" │ │ Confidence: "exact" │ │ STOP │ │ │ └─ NO → Status: "Numerical Error" │ Go to NUMERICAL ERROR DETAILS │ │ NUMERICAL ERROR DETAILS (Academic Precision Mode) │ ├─ Record: │ • Source value: [exact number from source] │ • Claimed value: [number in document being checked] │ • Deviation: [calculate exact difference] │ • Source location: [page, table, section] │ ├─ Classification: │ • ANY rounding → Numerical Error │ • ANY truncation → Numerical Error │ • Significant figures mismatch → Numerical Error │ • Unit mismatch → Numerical Error │ • Wrong direction (e.g., increase vs decrease) → Hallucination │ └─ Exception: If source ITSELF provides rounded figure • e.g., Source says "96.555% (approximately 97%)" • Then claiming "97%" → Verified (cite the approximation) Numerical Precision Rules (Academic Standard) Default mode: Strict academic precision. Exact numbers only. Rule Source Claim Status Exact match required 96.555% 96.555% ✓ Verified Any rounding = error 96.555% 97% ✗ Numerical Error Any rounding = error 96.555% 96.6% ✗ Numerical Error Truncation = error 96.555% 96.5% ✗ Numerical Error Sig figs must match 0.834 0.83 ✗ Numerical Error Units must match 96.555% 0.96555 ✗ Numerical Error Direction matters +12% growth +15% growth ✗ Hallucination Order of magnitude $4.7B $47B ✗ Hallucination Numerical Error Output Format ### Numerical Error: [Claim ID] | Field | Value | |-------|-------| | Claim | "Model achieves 97% accuracy" | | Location | Slide 4, bullet 2 | | Source | Chen et al. (2024), Table 3, p.8 | | Source value | 96.555% | | Claimed value | 97% | | Deviation | +0.445% (rounded up) | | Status | Numerical Error | | Fix | Replace with: "Model achieves 96.555% accuracy" | Confidence Classification Level Criteria Use when exact ≥95% word overlap OR identical number with identical units Direct quote, exact statistic paraphrase Same fact, different words, no interpretation added Restated finding interpretation Inference drawn from source data Calculated from source, synthesized Rule: When uncertain between levels, use the MORE CONSERVATIVE option and flag for review. Mandatory Search Templates Run ALL applicable templates. Do not stop after first result. For Academic Citations Query 1: "[first author last name] [year] [first 3 words of title]" Query 2: "[full paper title]" site:semanticscholar.org OR site:arxiv.org Query 3: "[first author] [year] [venue/journal name]" Query 4: "doi:[DOI]" (if DOI provided) Query 5: "arxiv:[arxiv_id]" (if arXiv ID provided) For Statistics (Market size, usage numbers, etc.) Query 1: "[exact number with unit] [topic] [year]" Query 2: "[topic] [year] statistics report site:statista.com" Query 3: "[topic] [year] report site:mckinsey.com OR site:gartner.com" Query 4: "[topic] market size [year] site:gov OR site:edu" Query 5: "[topic] [number] original source" For Company/Product Claims Query 1: "[company name] [claim topic] press release [year]" Query 2: site:[company domain] [claim topic] Query 3: "[company name] [metric] official announcement" Query 4: "[company name] [claim] SEC filing" (for public companies) For Health/Medical Claims Query 1: "[claim topic] site:who.int OR site:cdc.gov OR site:nih.gov" Query 2: "[claim] systematic review site:cochrane.org" Query 3: "[claim] meta-analysis pubmed" For Government/Policy Claims Query 1: "[policy/law name] site:gov" Query 2: "[statistic] official statistics [country]" Query 3: "[claim] [agency name] report" Source Authority Hierarchy When multiple sources found, prefer in this order: Rank Source Type Examples 1 Primary source Original study, official report, raw data 2 Government/institutional WHO, CDC, World Bank, national statistics offices 3 Peer-reviewed publication Nature, Science, IEEE, ACM 4 Industry reports (named) Gartner, McKinsey, Statista (with methodology) 5 Reputable news citing primary NYT, Reuters citing original source 6 Secondary compilations Wikipedia (check their sources) Rule: If only Rank 5-6 sources found, status = "Unverified" with note "Only secondary sources found" Multi-Source Verification (Search Mode) A claim achieves "Verified" status only if: Condition Sources Required Primary source found 1 (if authoritative: .gov, peer-reviewed, official) Only secondary sources ≥2 independent sources agreeing Sources conflict Status = "Unverified", note the conflict Tie-Breaker Rules When uncertain, apply these rules. No judgment calls. | Situation | Rule |