Daily harness signal

Generate agent docs—or delete the bundle

August 11, 2026 · JST One fresh finding Documentation · drift · CI
A machine-oriented documentation bundle is a cache, not a second source of truth. Generate it from canonical pages, then make CI fail on drift before agents ingest stale instructions.
01 · Fresh · source date 2026-08-10

Give agent-facing doc bundles a rebuild receipt

Use when: your repository ships complete.md, llms-full.txt, a generated manual, or any concatenated file that agents may load whole. Treat the risk as active whenever humans can edit that bundle directly, canonical pages move, links depend on source-relative paths, or schemas and command examples have their own release cadence.

Action: choose one canonical docs tree and mark the bundle generated. Add a deterministic make docs-bundle target that orders pages from an explicit manifest, rewrites or removes source-relative links, embeds the canonical schema or release identifier, and writes the agent feed. Add make check-docs-bundle to CI; it must regenerate into a temporary path, compare bytes with the published artifact, validate every link, and reject references to retired paths. Route all fixes to canonical pages first. If no consumer needs the bundle, delete it instead of maintaining a duplicate.

Acceptance check: on a clean checkout, make docs-bundle && git diff --exit-code and make check-docs-bundle must pass. Then change one canonical sentence, rename one linked page, and advance a schema fixture without rebuilding. The check must fail for all three canaries. Regenerate; require a clean diff check, zero unresolved links, the current schema identifier, and inclusion of every manifest page. A documentation bug fixed only in the generated file is an automatic failure.

Evidence: the official MCP Registry merged PR #1522 after auditing its 2,944-line complete.md: it represented 31% of documentation, had no generator, all 72 relative links were broken, roughly 14 current pages were missing, and examples used a superseded schema. A prior user-reported CLI flag correction had landed only in that dead bundle. The verified merge removed 2,945 lines and checked all 101 remaining relative links. OpenAI’s current Codex docs provide the counterexample: an index points to an explicitly generated single-file export.

Caveat: do not infer that full-corpus feeds are inherently bad. They are useful for long-context or ingestion workflows when produced from the same sources on every deploy. Byte parity and link checks still cannot prove semantic or runtime truth; retain executable schema/API checks and dated staleness banners. If the bundle is build-only rather than committed, compare the deployed artifact against a fresh build instead of using git diff.

Compact source notes

  1. MCP Registry PR #1522 (2026-08-10). Merged official repository audit: source inventory, broken-link counts, missing-page list, stale-schema example, proposed generator/check shape, and post-repair link verification.
  2. Verified merge commit a25f166 (2026-08-10). Inspectable +54/−2,956 diff across 12 files, including deletion of the 2,945-line duplicate.
  3. MCP Registry issue #767 (2025-11-13). Minimal reproduction for the publisher’s ignored --file flag; PR #1522 documents how a later correction was misdirected into the dead bundle rather than canonical docs.
  4. OpenAI Codex documentation index and single-file export (retrieved 2026-08-11). Current official evidence that full agent feeds remain useful when explicitly generated from a documentation set.
  5. Method: anchor lens—merged diff, exact file/link/page counts, issue reproduction, and public generated counterexample; unity lens—the human site and machine feed are two views of one documentation authority. A direct-edit path or negative canary that leaves CI green falsifies the control. Confidence: Confirmed for the MCP incident; Likely for the generalized gate.