The open source agent readiness library

A person and an AI crawler do not arrive at your site the same way. A person opens a browser, clicks through to pricing, and reads that page with the layout on screen. An AI crawler, or an agent that will cite the page in an answer, requests one URL and reads the HTML that comes back. The clicks are not in that response.

Making the site visible to that crawler is a different job from making the page clear to a person. The page it has to use is the pricing page, a documentation page, or one URL from a template that generates hundreds of pages, and it is rarely the homepage. For the crawler, the page is visible when the crawler is allowed to fetch it, a parser can lift the claim out of the navigation and the footer, and the page gives a reason to trust the claim. llms.txt and robots.txt at the site root can both look fine while the pricing page is blocked, or while it has no sentence a model can quote.

The homepage file is not the page

A pass on one question does not cover the other two.

• Reach. Can the crawler fetch the page, and is the page eligible to be quoted? robots.txt rules for GPTBot, ClaudeBot, and PerplexityBot, plus noindex, nosnippet, and max-snippet in the meta robots tag and X-Robots-Tag. Google's own AI features guide treats snippet eligibility as the hard gate for AI Overviews and AI Mode.
• Read. Can a parser lift the claim? JSON-LD that parses, one h1, a content-to-boilerplate ratio, definition blocks, and FAQ markup where a FAQ page is actually a FAQ.
• Cite. Does the page give a reason to trust the claim? Author and trust links, outbound citations, a business name that matches the page, and a date that says the page was touched.

I maintain @canonry/agent-readiness. Version 8.0.1 scores those three questions as 16 factors. The weights are relative. The core set sums to 111, not 100, and each factor reports its real share as sharePct. A factor the analyzer marks not applicable is dropped, weight and all, so a product page is not punished for having no FAQ. The CLI exits 0 at an overall score of 70 or above and 1 below.

Two checks exist because "the file returned 200" is a weak observation. A content-negotiation probe asks for llms.txt again with Accept: text/markdown. If the bare request is fine and the markdown request is not, the finding is a content-negotiation trap. Astro, Vercel, and Starlight sites do this when they redirect .txt to a .md path that is not there. The file looks present to a browser and is missing for the client that asked for markdown. A sitemap or llms.txt that answers 200 with an HTML shell is a soft 404, and the finding names where the redirect went.

Google's guide also says llms.txt, and a pile of structured data, are not required for Google's AI features. The scorer still counts them, because Google is one engine. Snippet eligibility stays the factor that tracks the gate Google does enforce.

A factor that does not apply is left out

Site averages lie in a predictable way. An FAQ that lives on 8 of 500 pages averages near zero if you divide by every URL. The number is true and useless. That site was never supposed to be one FAQ.

Sitemap aggregation keeps two means. avgScore divides by every audited page. applicableAvgScore divides by the pages the factor applies to. FAQ Content and Definition Blocks report applicability from schema, URL, title, and page structure. A silent analyzer keeps the older rule: a site-wide factor always applies, and a page-specific factor applies where its score is at least 30.

Each rolled-up issue also carries a status:

A limited FAQ is a tune-up of /faq, scoped to the pages that carry it. It is not an order to paste FAQ markup into every property URL. Presence is the gate, not coverage. Thin coverage is the normal state for those two factors.

The same honesty applies inside a factor. entity-consistency.schema-identity.mismatch fires when LocalBusiness JSON-LD, including a subtype such as Hotel, names a business the page never shows, or a country the page never mentions. It skips a bare Organization, a generic name, and a page with no title or h1 and fewer than 50 words. The code does not change the score. It leads that factor's recommendation list. A title shorter than 15 characters, entity-consistency.title.short, is the same kind of note: advisory, not a penalty.

Critical defects go the other direction. A homepage with four h1 elements, or a contact page with none, is a one-line structural break. Averaged into Technical SEO it can disappear under a passing factor score, because prevalence ranking cares how many pages are affected. criticalDefects lists those binary breaks by impact instead. The offending URLs are named, homepage and high sitemap priority first, and the critical ones lead prioritizedFixes. The stable code on the finding is technical-seo.h1.multiple, not a sentence the next release might reword.

Pages that share a template are one fix

A 400-URL property site is not 400 audits. /properties/austin/the-mark and /properties/denver/the-mark are one template until a page proves otherwise. The library infers those segments from the sitemap, keeps the homepage, and round-robins a budget of 200 across the templates. It also cycles the first identifier segment, so a property budget does not all land on the city that sorts first.

coverage.confidence is full, representative, or indicative, and it keys off template reach rather than the raw percentage. A 5 percent sample that touched every URL shape generalizes. A 45 percent prefix that never left one section does not. The sample is stratified across templates. It is not the first rows of the sitemap.

templateGroups then collapses pages that share a URL shape and score alike. Each group has a templateKey, a pageCount, an avgScore, a representativeUrl, and the full URL list. A fix carries templateCount and instanceCount, so "194 property pages missing schema" is priced as the one template edit it is. Per-page results stay complete. Nothing is dropped from affectedPages to make the rollup tidy.

schemaVersion versions that JSON shape apart from the npm version. Pin the field. Treat a major bump as a break, and treat a missing field as a report from before 2.0. --format agent is the smaller contract: schemaVersion, the score, a pass bit, criticalDefectCount, and the ranked issues. The agent does not average factor scores itself. Full --format json remains the report you store.

runSiteCrawl answers the question a sitemap cannot: can a crawler reach the page by following links? It starts at the root, reads recursive sitemaps and the Sitemap directives in robots.txt, then follows internal HTML links on that exact host. Cross-host links are recorded and never followed. A root that redirects off-host comes back partial, with terminationReason: 'root-host-redirect'.

Each anchor edge records where it sat, from HTML landmarks rather than class names. A nav inside main is navigation. A header inside an article is the post, not the site chrome. unknown means the page gave no landmark that answers. A page linked only from the footer is a footer link. A page with no followable link is not a page the crawl could reach.

Broken and unchecked links stay in different arrays. deadLinks.findings answered 4xx or 5xx. A 429 is throttled inside deadLinks.unverified, because that status describes our request rate. The engine retries a transient failure before it classifies the row, and it does not fetch external links for this check.

Private and loopback addresses stay blocked. --allow-local relaxes that for the single host you named. A sitemap entry or a redirect that points at 169.254.169.254, or at any other private host, is still refused. There is no flag that turns the guard off.

The build is the thing CI can trust

A live URL changes under you. A bot wall, a timeout, or a sitemap that grew since yesterday moves the score without a commit. The static entry point refuses that variable.

runStaticAudit('./out', { baseUrl: 'https://example.com' }) reads the built HTML and does no network I/O. A directory aggregates like a sitemap run. index.html maps to its directory URL. llms.txt, llms-full.txt, robots.txt, and sitemap.xml are read from the directory root when they are there. The same commit produces the same score, because nothing else is consulted.

Coverage is partial, and the API docs say so. Redirects, X-Robots-Tag, Last-Modified, and Link headers are not in the files. Factors that need those headers score as if the header were absent. Audit the deployed URL when those signals are the question. Audit the build when the question is whether this pull request changed the HTML.

compare is a separate command. It reads two JSON reports and touches no network. A regression is any of these: the aggregate score drops by more than 2 points, one page drops by more than 5, one factor drops by more than 8, a page that was auditing now errors, a new severity: critical defect appears, or the report schema takes a major bump. Those tolerances are flags. Score deltas gate only when the two runs are comparable, meaning the same factor set and no major engine change. Otherwise the diff warns. --strict-comparability turns that mismatch into exit 2, which is what you want for a baseline stored as a CI artifact. The first run with no baseline passes, so a new workflow can land before someone seeds the file.

That is a different gate from "fail under 85." A floor fails when the network blips, and it passes a real regression that stays above the line. A new second h1 in a shared template trips --fail-on-new-critical even when the aggregate move is inside the 2-point tolerance. missing-meta-description is a warning, so it does not trip that gate. Use --require-meta on the audit when a missing description should fail the job.

The Agent Readiness Guard is that compare, wrapped so GitHub Actions builds the site, audits ./out, and posts the diff on the pull request. The action is Canonry/agent-readiness/action@v8. --changed is the smaller cut of the same idea: it reads git diff against a base ref, infers static routes, and audits those sitemap URLs. A dynamic segment such as [slug] is skipped, because the file path does not name the URL. Pass --include-critical when the homepage still has to be in the run.

The other libraries are better at the jobs next door

I compared the libraries a team will actually find in November 2026. They do not fail in the same way, so a single "higher score" column would be a trick.

readiness-audit 2.6.0 is the strongest standard in the set. The maintainers published a week of Cloudflare logs across 56 domains: bots in Cloudflare's AI categories requested robots.txt 3,644 times, the sitemap 2,232 times, and llms.txt 49 times. The same week saw 12,601 requests for markdown at ordinary page URLs, 92 percent of them from Exa's crawler. That is why their required items follow evidence, and why an item that only scanners look for is recommended at most. It is also why their D12 check matters: an OAuth, A2A, plugin, or server-card claim with no service behind it fails. An A2A card that is not a valid A2A card fails D11. Declining an item in writing is fine. Faking a pass is not.

Their CLI also does work people assume only a larger library would. --base http://localhost:3000 scores a preview by fetching your server. --compare lists what changed since a saved report. The process exits 1 when a required check fails. It has no dependencies, and it pins Node to current security releases on the 22, 24, and 26 lines. The standard marks what a scorecard cannot see. T5 is the item I will not pretend to cover: hand a fresh agent only the domain and a goal, and record whether it succeeds.

Use readiness-audit when the question is whether the origin publishes the discovery files the standard can verify, and whether those declarations are true. Keep it beside this library. A 40-page sample will not see instance four of a broken template, and a file check will not notice Hotel JSON-LD that names a sister property.

a14y is the readability scorecard, from the Vercel agent readability notes, Apache-2.0, with a CLI, @a14y/core, and a Chrome extension. Scorecards are versioned: one scan can be graded as 0.2.0 and as draft. That is a cleaner rubric move than a silent npm major. --output agent-prompt is a fix list for a coding agent, and --fail-under is an absolute floor. The project reports a study in which raising one site from 37 to 89 cut Claude's token use by 49 percent and its tool calls by 52 percent, with answer quality unchanged. I have not reproduced that study. If you want cheaper reading of a docs site, start there. The floor still passes a regression that stays above it, and the published command audits a URL rather than a directory of HTML you have not served.

Agent Lighthouse is the wide live scan, and the core, the HTML report, and the MCP server are separate packages. That split is the better boundary if you only want to render a report. The scorer returns no number when the scan never reached the site, or when more than 35 percent of the assessed weight went unread. Page-type skips do not count toward that threshold. Their GitHub Action takes a URL and a minimum score, which is the right gate for a staging origin and the wrong gate for a template change in a commit. WebMCP, OpenAPI, agents.json, and commerce offers sit in their catalog. In ours they are behind --include-agent-skills, and the default 16 factors do not grade a checkout.

@advance-labs/aeo-toolkit is the kit to import when you are building an engine rather than calling one. @advance-labs/net-guard, the crawler, the HTML parser, and the schema validator are separate Apache-2.0 packages. On 20 October 2026 the hosted console, OAuth, and the three MCP servers moved to a private repository. The open half is a 44-check scorer plus those parts. Take the SSRF-safe fetch if that is the hole in your own crawler. Take @canonry/agent-readiness if you want prioritizedFixes without assembling the pipeline.

agent-readiness-auditor answers a question this library does not. Its heaviest check is hidden prompt-injection content, on the argument that a readable page which hijacks an agent is worse than a page which is merely hard to read. Nine checks, a letter grade, the landing page plus a few key pages. I agree with the threat model and the library does not implement it. Run that auditor when safety is the score. Do not read our overall score as a claim that the HTML is safe to give an agent.

Lighthouse remains the instrument for performance, accessibility, and best practices. --lighthouse calls PageSpeed Insights for those three scores on one URL, and it cannot run with a sitemap audit because each call takes 15 to 30 seconds. It stays off by default so the build path needs no quota. Recommendations also cite the matching rule in the specification.website agent readiness checklist. That site is the broader spec. This library scores the reach, read, and cite slice, then diffs it.

Import it when the page has to stay citeable

The decision is the entry point, not the badge count.

Install @canonry/agent-readiness on Node 20 or later. No API key. The license is MIT. The package used to be @canonry/aeo-audit. The 8.0 line, released 7 November 2026, is the rename and the current report shape. In the sample below, previousReport is the last directory or sitemap JSON you saved.

import { compareReports, runStaticAudit, type SitemapAuditReport } from '@canonry/agent-readiness'

const baseline: SitemapAuditReport = previousReport
const result = await runStaticAudit('./out', {
  baseUrl: 'https://example.com',
})

if (result.kind === 'multi') {
  const diff = compareReports(baseline, result.report)
  if (diff.verdict === 'fail') throw new Error(diff.failReasons.join('\n'))
}

compareReports is the function the CLI calls. A directory audit returns kind: 'multi'. verdict is fail when a gating dimension tripped, and failReasons is the list CI should print. The shell form is the same diff:

npx @canonry/agent-readiness ./out --base-url https://example.com --format json > current.json
npx @canonry/agent-readiness compare --baseline baseline.json --current current.json --strict-comparability

For a live site, runSitemapAudit is the sample and runSiteCrawl is the link graph. runAgentReadinessAudit fetches the one URL you pass. Pass factors when you want a subset. Schema validity alone catches a duplicate FAQPage and a JSON-LD block that does not parse.

The same engine is what Canonry runs for Site Health: a scheduled crawl, a stored run, and the internal link map beside mention and citation tracking. You do not need the platform to use the library.

Keep readiness-audit when you want the log-grounded standard and the false-declaration checks. Keep a14y when you are pinning a readability scorecard, especially on docs. Keep Agent Lighthouse when the surface is an API, an MCP server, or a checkout. Keep Lighthouse when the regression is latency. None of those replace the report you diff against last week's HTML. This library does not replace the evidence, the readability scorecard, or the safety check they are better at.