Technical SEO audit
Can search engines and AI crawlers actually index this page? One pass over reachability, TLS, robots policy, sitemap health, meta/OpenGraph tags, and the on-page link graph.
When to use this pack
You shipped a page (or inherited a site) and want the technical indexability picture without opening six tools: is it up and fast, is TLS valid, does robots.txt allow the crawlers that matter (including LLM bots), does the sitemap parse, are title/description/OG tags present and sized right, and where do its links point? Distinct from a security audit (headers/SPF/CT) and an uptime snapshot — this is the 'will Google and ChatGPT see what I meant' check. Run it pre-launch, post-migration, or on a competitor.
Tools in this pack
-
HTTP check
$0.003
POST /api/http-check
Check any public URL: status code, latency, final URL after redirects, and response headers. The uptime primitive for agent monitors.
-
TLS certificate
$0.003
POST /api/tls-cert
Inspect the TLS certificate of any public host: subject, issuer, validity window, days remaining, SANs, and SHA-256 fingerprint.
-
Robots.txt check
$0.002
POST /api/robots-check
Fetch a site's robots.txt and answer: may this user-agent crawl this path? Returns the matched rule and all declared sitemaps.
-
Sitemap reader
$0.003
POST /api/sitemap
Fetch and parse a sitemap.xml (or sitemap index): returns up to 500 URLs with lastmod, or the child sitemaps of an index.
-
Page metadata
$0.002
undefined
Fetch page metadata for a URL: title, description, OpenGraph, Twitter cards, canonical URL, favicon.
-
HTTP headers + security analysis
$0.003
POST /api/http-headers
Fetch a URL and return every response header plus a security analysis: HSTS, CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, COOP/CORP/COEP. Scores 0–100 by presence, flags weak HSTS, and warns on Server/X-Powered-By identity leaks. SSRF-protected.
Workflow
- http-check with the page URL — status, latency, redirect chain. A 200 under ~800ms is healthy; a 3xx chain longer than one hop wastes crawl budget and should be flattened.
- tls-cert with the host — issuer, expiry, chain trust. daysRemaining < 21 is a renewal warning; an untrusted chain is an indexing risk (and a browser warning for humans).
- robots-check with the page URL — is it fetchable by crawlers? Re-run with userAgent values you care about (Googlebot, GPTBot, ClaudeBot, PerplexityBot) — LLM crawler policy is part of modern SEO, and an accidental Disallow: / for AI bots silently removes you from answer engines.
- sitemap with the site's sitemap.xml URL — type, URL count, parse errors. A sitemap that 404s or is empty means discovery depends entirely on the link graph.
- meta with the page URL — title, description, canonical, OpenGraph/Twitter cards. Flag: missing description, title > 60 chars, missing og:image (kills social/link-preview CTR), missing canonical on parameterized URLs.
- http-headers with the page URL — the response headers search engines act on. Flag: an X-Robots-Tag: noindex (silently removes the page from every index regardless of robots.txt), a missing/short Cache-Control on static assets, and a Content-Type without charset. Summarize as a pass/warn/fail card per check with the two highest-impact fixes on top.
Run it in Claude
claude mcp add agent402 -s user -- npx -y agent402-mcp@latest
Then paste this prompt into Claude:
Run a technical SEO audit of https://example.com with Agent402: (1) http-check {url} — status + latency + redirects. (2) tls-cert {host: "example.com"} — expiry + trust. (3) robots-check {url, userAgent: "Googlebot"} and again with "GPTBot" — crawlability for search AND answer engines. (4) sitemap {url: "https://example.com/sitemap.xml"} — parses, URL count. (5) meta {url} — title/description/OG completeness, title ≤ 60 chars. (6) http-headers {url} — flag X-Robots-Tag: noindex and cache policy. Return a card: {reachability, tls, robots: {googlebot, gptbot}, sitemap, meta, headers, topFixes: [two highest-impact items]}.← All skill packs