citecheck

name citecheck version 0.1.0 description Use when the user asks to "verify citations", "check references", "validate paper citations", or "evaluate reference relevance". It extracts references from LaTeX/PDF papers, checks formatting rules, verifies existence via Crossref / Semantic Scholar / OpenAlex / PubMed / arXiv / dblp / Google Scholar / WebSearch, and scores thematic/semantic relevance. CiteCheck — Paper Citation Verification Overview CiteCheck verifies academic paper citations by combining structured parsing with agent-native LLM evaluation. It supports LaTeX source files (preferred) and PDF fallback. Workflow Parse paper : Call citecheck CLI to read LaTeX (preferred) or PDF, extract references and body text Format check : Call citecheck CLI to validate bibliography entries Queryability verification : Call citecheck CLI to verify existence via Crossref / Semantic Scholar Evaluate thematic relevance : Evaluate directly using the agent's reasoning capabilities — compare cited paper title/abstract/venue against the citing paper Evaluate semantic accuracy : Evaluate directly using the agent's reasoning capabilities — compare in-text citation context against cited source content Generate report : Aggregate all results into a Markdown report Why matching steps are not done by the CLI The citecheck CLI can run standalone with optional --api-key for external LLM-powered matching. When used as a Skill, the host agent itself possesses LLM reasoning capabilities. Direct evaluation is faster, more consistent, and requires no additional API keys from the user . 1. Parse Paper (CLI) LaTeX source (preferred) citecheck path/to/latex_project/ --skip-verification --skip-semantic -o parsed_report.md Or parse a single file: citecheck main.tex --skip-verification --skip-semantic -o parsed_report.md PDF (fallback) citecheck paper.pdf --skip-verification --skip-semantic -o parsed_report.md --skip-verification and --skip-semantic are required in Skill mode because steps 3–5 are performed directly by the agent. 2. Format Check (CLI) Call citecheck to check and report format issues: Check item Description Required fields Author, title, year, venue completeness Format consistency Punctuation, capitalization, abbreviation uniformity DOI/URL If present, whether format is correct and accessible Year sanity No 202x placeholders, not in the future Detailed rules: see ./references/format-check-rules.md . 3. Queryability Verification (CLI + Agent Supplement) Call citecheck to query Crossref and Semantic Scholar public APIs for citation existence. API details: see ./references/api-reference.md . If the CLI fails due to network/SSL issues , use WebSearch to directly query suspicious citations (especially those marked "unverifiable"), supplementing the verification results. 4. Evaluate Thematic Relevance (Agent Direct) Do not call the CLI or any external API. Evaluate directly using the agent's reasoning capabilities. For each reference, extract: Citing paper: title , abstract , keywords Cited paper: title , abstract (from API results or WebSearch), venue Evaluation prompt template: see ./references/thematic-scoring-prompt.md . 5. Evaluate Semantic Accuracy (Agent Direct) Do not call the CLI or any external API. Evaluate directly using the agent's reasoning capabilities. For each in-text citation position: Extract 1–2 sentences before and after the citation marker as citing_text Obtain the cited paper's abstract via Semantic Scholar or WebSearch as cited_text Evaluate semantic consistency using the prompt template: see ./references/semantic-matching-prompt.md . 6. Output Report Generate a Markdown report containing: Summary Total references Format issues count Query failures / suspicious count Average thematic relevance score Average semantic accuracy score Detailed Results Table No. Title Format Queryable Thematic Semantic Notes 1 ... ✅/⚠️/❌ ✅/❌ 0.85 0.90 DOI mismatch Issue Summary List all findings and recommendations by severity Dependencies pip install CiteCheck For PDF support: pip install CiteCheck[pdf] Notes No --api-key is required in Skill mode . Thematic and semantic matching are performed directly by the agent, with no external LLM API calls. Prioritize LaTeX source files; parsing accuracy is far higher than PDF. Add delays and retries to API calls to avoid rate limiting. When Semantic Scholar abstracts are missing, fall back to Crossref + title keywords. Semantic matching requires the cited paper's abstract; mark "source unreachable" when unavailable. If placeholder citations like =? or [?, ?] are found, mark them as "unverifiable references".