Zillowzillow.com
Score breakdown
Evidence
- ✕Homepage blocks identified agents: ChatGPT-User (HTTP 403), Claude-User (HTTP 403), Perplexity-User (HTTP 403), GoogleAgent-Mariner (HTTP 403)
- ✓robots.txt has no rule against any user-directed agent we test
- ◐Also disallows 4 of 15 training/search crawlers (not scored; agents ≠ crawlers)
- ◐No Content-Signal in robots.txt
- ◐No llms.txt
- ◐No JSON-LD structured data on the homepage
- ◐Missing main/nav landmarks
- ◐No agent-protocol manifest (UCP, MCP, A2A, OpenAPI)
| Identified as | HTTP | Result |
|---|---|---|
ChatGPT-User | 403 | blocked · “Access to this page has been denied” |
Claude-User | 403 | blocked · “Access to this page has been denied” |
Perplexity-User | 403 | blocked · “Access to this page has been denied” |
GoogleAgent-Mariner | 403 | blocked · “Access to this page has been denied” |
browser (human) | 200 | normal page |
What Zillow would need to change
1Verify agents cryptographically instead of blocking them at the doorAccess · up to 23 pts
Your edge answers ChatGPT-User, Claude-User, Perplexity-User, GoogleAgent-Mariner with a block or interstitial. Web Bot Auth (HTTP Message Signatures, RFC 9421) lets an agent prove who it is per request; Cloudflare, Akamai, AWS WAF and Vercel can verify it. Allow verified agents and keep challenging the rest.
# Cloudflare: Security → Bots → Verified bots → allow category "AI Agent" # Or verify yourself: fetch the agent's key from # https://<signature-agent>/.well-known/http-message-signatures-directory # and check the Signature / Signature-Input headers (tag="web-bot-auth")
2Give agents a front doorProtocols · up to 12 pts
A Universal Commerce Protocol manifest for shopping, or an MCP server card for everything else. An agent that can call an API never needs to hold a customer's password.
GET /.well-known/ucp
{ "ucp": { "version": "2026-01-23",
"services": { "dev.ucp.shopping": [{ "transport": "rest", "endpoint": "https://www.zillow.com/api/ucp/v1" }] },
"capabilities": { "dev.ucp.shopping.checkout": [ … ] } } }
# or
GET /.well-known/mcp.json
{ "name": "Zillow", "endpoint": "https://www.zillow.com/mcp", "transport": "streamable-http" }3Publish llms.txtSurface · up to 8 pts
A short markdown map of what the site does, where the important pages are, and how to search or check out.
/llms.txt # Zillow > One paragraph on what this site is for. - [Products](/products): how to search and filter - [Checkout](/checkout): steps and constraints
4Add JSON-LD structured dataSurface · up to 6 pts
schema.org Organization on the homepage, Product + Offer on product pages, with price and availability.
<script type="application/ld+json">
{ "@context": "https://schema.org", "@type": "Organization", "name": "Zillow", "url": "https://www.zillow.com/" }
</script>5Use semantic landmarksSurface · up to 5 pts
main, nav, header, footer and labeled forms. Everything an accessibility audit already asks for.
<header>…</header> <nav aria-label="Primary">…</nav> <main>…</main> <form aria-label="Site search"><label for="q">Search</label><input id="q" name="q"></form>
6Say your stance with a Content-SignalConsent · up to 3 pts
Two lines in robots.txt end the guessing game. ai-input covers agents reading pages on a person's behalf.
User-Agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: /