ln-614-docs-fact-checker

Extracts verifiable claims from ALL .md files (paths, versions, counts, configs, names, endpoints), verifies each against codebase, cross-checks between documents for contradictions.

Safety Notice

This listing is imported from skills.sh public index metadata. Review upstream SKILL.md and repository scripts before running.

Copy this and send it to your AI assistant to learn

Install skill "ln-614-docs-fact-checker" with this command: npx skills add levnikolaevich/claude-code-skills/levnikolaevich-claude-code-skills-ln-614-docs-fact-checker

Paths: File paths (shared/, references/, ../ln-*) are relative to skills repo root. If not found at CWD, locate this SKILL.md directory and go up one level for repo root.

Documentation Fact-Checker (L3 Worker)

Specialized worker that extracts verifiable claims from documentation and validates each against the actual codebase.

Purpose & Scope

  • Worker in ln-610 coordinator pipeline - invoked by ln-610-docs-auditor
  • Extract all verifiable claims from ALL .md files in project
  • Verify each claim against codebase (Grep/Glob/Read/Bash)
  • Detect cross-document contradictions (same fact stated differently)
  • Includes docs/reference/, docs/tasks/, tests/ in scope
  • Single invocation (not per-document) — cross-doc checks require global view
  • Does NOT check scope alignment or structural quality

Inputs (from Coordinator)

MANDATORY READ: Load shared/references/audit_worker_core_contract.md.

Receives contextStore with: tech_stack, project_root, output_dir.

Workflow

Phase 1: Parse Context

Extract tech stack, project root, output_dir from contextStore.

Phase 2: Discover Documents

Glob ALL .md files in project. Exclude:

  • node_modules/, .git/, dist/, build/
  • docs/project/.audit/ (audit output, not project docs)
  • CHANGELOG.md (historical by design)

Phase 3: Extract Claims (Layer 1)

MANDATORY READ: Load shared/references/two_layer_detection.md for detection methodology.

For each document, extract verifiable claims using Grep/regex patterns.

MANDATORY READ: Load references/claim_extraction_rules.md for detailed extraction patterns per claim type.

9 claim types:

#Claim TypeWhat to ExtractExtraction Pattern
1File pathsPaths to source files, dirs, configsBacktick paths, link targets matching src/, lib/, app/, docs/, config/, tests/
2VersionsPackage/tool/image versionsSemver patterns near dependency/package/image names
3Counts/StatisticsNumeric claims about codebase`\d+ (modules
4API endpointsHTTP method + path`(GET
5Config keys/env varsEnvironment variables, config keys[A-Z][A-Z_]{2,} in config context, process.env., os.environ
6CLI commandsShell commandsnpm run, python, docker, make in backtick blocks
7Function/class namesCode entity referencesCamelCase/snake_case in backticks or code context
8Line number refsfile:line patterns[\w/.]+:\d+ patterns
9Docker/infra claimsImage tags, ports, service namesImage names with tags, port mappings in docker context

Output per claim: {doc_path, line, claim_type, claim_value, raw_context}.

Phase 4: Verify Claims (Layer 2)

For each extracted claim, verify against codebase:

Claim TypeVerification MethodFinding Type
File pathsGlob or ls for existencePATH_NOT_FOUND
VersionsGrep package files (package.json, requirements.txt, docker-compose.yml), compareVERSION_MISMATCH
CountsGlob/Grep to count actual entities, compare with claimed numberCOUNT_MISMATCH
API endpointsGrep route/controller definitionsENDPOINT_NOT_FOUND
Config keysGrep in source for actual usageCONFIG_NOT_FOUND
CLI commandsCheck package.json scripts, Makefile targets, binary existenceCOMMAND_NOT_FOUND
Function/classGrep in source for definitionENTITY_NOT_FOUND
Line numbersRead file at line, check content matches claimed contextLINE_MISMATCH
Docker/infraGrep docker-compose.yml for image tags, portsINFRA_MISMATCH

False positive filtering (Layer 2 reasoning):

  • Template placeholders ({placeholder}, YOUR_*, <project>, xxx) — skip
  • Example/hypothetical paths (preceded by "e.g.", "for example", "such as") — skip
  • Future-tense claims ("will add", "planned", "TODO") — skip or LOW
  • Conditional claims ("if using X, configure Y") — verify only if X detected in tech_stack
  • External service paths (URLs, external repos) — skip
  • Paths in SCOPE/comment HTML blocks describing other projects — skip
  • .env.example values — skip (expected to differ from actual)

Phase 5: Cross-Document Consistency

Compare extracted claims across documents to find contradictions:

CheckMethodFinding Type
Same path, different locationsGroup file path claims, check if all point to same real pathCROSS_DOC_PATH_CONFLICT
Same entity, different versionGroup version claims by entity name, compare valuesCROSS_DOC_VERSION_CONFLICT
Same metric, different countGroup count claims by subject, compare valuesCROSS_DOC_COUNT_CONFLICT
Endpoint in spec but not in guideCompare endpoint claims across api_spec.md vs guides/runbookCROSS_DOC_ENDPOINT_GAP

Algorithm:

claim_index = {}  # key: normalized(claim_type + entity), value: [{doc, line, value}]
FOR claim IN all_verified_claims WHERE claim.verified == true:
  key = normalize(claim.claim_type, claim.entity_name)
  claim_index[key].append({doc: claim.doc_path, line: claim.line, value: claim.claim_value})

FOR key, entries IN claim_index:
  unique_values = set(entry.value for entry in entries)
  IF len(unique_values) > 1:
    CREATE finding(type=CROSS_DOC_*_CONFLICT, severity=HIGH,
      location=entries[0].doc + ":" + entries[0].line,
      issue="'" + key + "' stated as '" + val1 + "' in " + doc1 + " but '" + val2 + "' in " + doc2)

Phase 6: Score & Report

MANDATORY READ: Load shared/references/audit_worker_core_contract.md and shared/references/audit_scoring.md.

Calculate score using penalty formula. Write report.

Audit Categories (for Checks table)

IDCheckWhat It Covers
path_claimsFile/Directory PathsAll path references verified against filesystem
version_claimsVersion NumbersPackage, tool, image versions against manifests
count_claimsCounts & StatisticsNumeric assertions against actual counts
endpoint_claimsAPI EndpointsRoute definitions against controllers/routers
config_claimsConfig & Env VarsEnvironment variables, config keys against source
command_claimsCLI CommandsScripts, commands against package.json/Makefile
entity_claimsCode Entity NamesFunctions, classes against source definitions
line_ref_claimsLine Number Referencesfile:line against actual file content
cross_docCross-Document ConsistencySame facts across documents agree

Severity Mapping

Issue TypeSeverityRationale
PATH_NOT_FOUND (critical file: CLAUDE.md, runbook, api_spec)CRITICALSetup/onboarding fails
PATH_NOT_FOUND (other docs)HIGHMisleading reference
VERSION_MISMATCH (major version)HIGHFundamentally wrong
VERSION_MISMATCH (minor/patch)MEDIUMCosmetic drift
COUNT_MISMATCHMEDIUMMisleading metric
ENDPOINT_NOT_FOUNDHIGHAPI consumers affected
CONFIG_NOT_FOUNDHIGHDeployment breaks
COMMAND_NOT_FOUNDHIGHSetup/CI breaks
ENTITY_NOT_FOUNDMEDIUMConfusion
LINE_MISMATCHLOWMinor inaccuracy
INFRA_MISMATCHHIGHDocker/deployment affected
CROSS_DOC_*_CONFLICTHIGHTrust erosion, contradictory docs

Output Format

MANDATORY READ: Load shared/references/audit_worker_core_contract.md and shared/templates/audit_worker_report_template.md.

Write report to {output_dir}/614-fact-checker.md with category: "Fact Accuracy" and checks: path_claims, version_claims, count_claims, endpoint_claims, config_claims, command_claims, entity_claims, line_ref_claims, cross_doc.

Return summary to coordinator:

Report written: docs/project/.audit/ln-610/{YYYY-MM-DD}/614-fact-checker.md
Score: X.X/10 | Issues: N (C:N H:N M:N L:N)

Critical Rules

MANDATORY READ: Load shared/references/audit_worker_core_contract.md.

  • Do not auto-fix: Report violations only; coordinator aggregates for user
  • Code is truth: When docs contradict code, document is wrong (unless code is a bug)
  • Evidence required: Every finding includes verification command used and result
  • No false positives: Better to miss an issue than report incorrectly. When uncertain, classify as LOW with note
  • Location precision: Always include file:line for programmatic navigation
  • Broad scope: Scan ALL .md files — do not skip docs/reference/, tests/, or task docs
  • Cross-doc matters: Contradictions between documents erode trust more than single-doc errors
  • Batch efficiently: Extract all claims first, then verify in batches by type (all paths together, all versions together)

Definition of Done

MANDATORY READ: Load shared/references/audit_worker_core_contract.md.

  • contextStore parsed successfully (including output_dir)
  • All .md files discovered (broad scope)
  • Claims extracted across 9 types
  • Each claim verified against codebase with evidence
  • Cross-document consistency checked
  • False positives filtered via Layer 2 reasoning
  • Score calculated using penalty algorithm
  • Report written to {output_dir}/614-fact-checker.md (atomic single Write call)
  • Summary returned to coordinator

Reference Files

  • Audit output schema: shared/references/audit_output_schema.md
  • Detection methodology: shared/references/two_layer_detection.md
  • Claim extraction rules: references/claim_extraction_rules.md

Version: 1.0.0 Last Updated: 2026-03-06

Source Transparency

This detail page is rendered from real SKILL.md content. Trust labels are metadata-based hints, not a safety guarantee.

Related Skills

Related by shared tags or category signals.

Coding

ln-782-test-runner

No summary provided by upstream source.

Repository SourceNeeds Review
Coding

ln-140-test-docs-creator

No summary provided by upstream source.

Repository SourceNeeds Review
Coding

ln-150-presentation-creator

No summary provided by upstream source.

Repository SourceNeeds Review
Coding

ln-110-project-docs-coordinator

No summary provided by upstream source.

Repository SourceNeeds Review