OpenTableopentable.com
Score breakdown
Evidence
- ✕robots.txt disallows 16 of 19 user-directed agents: GoogleAgent-Mariner, GoogleAgent-Shopping, Manus-User, MistralAI-User, Devin, Copilot, GrokBot, Grok-DeepSearch…
- ✕Homepage returns a bot interstitial (HTTP 403, “Access Denied”) to every non-browser client, agents included
- ◐robots.txt restricts some paths for ChatGPT-User, Claude-User, Perplexity-User
- ◐robots.txt names 31 user-agents
- ◐Also disallows 11 of 15 training/search crawlers (not scored; agents ≠ crawlers)
- ◐No Content-Signal in robots.txt
- ◐No llms.txt
- ◐No agent-protocol manifest (UCP, MCP, A2A, OpenAPI)
| Identified as | HTTP | Result |
|---|---|---|
ChatGPT-User | — | no response |
Claude-User | — | no response |
Perplexity-User | — | no response |
GoogleAgent-Mariner | — | no response |
browser (human) | 403 | interstitial · “Access Denied” |
What OpenTable would need to change
1Verify agents cryptographically instead of blocking them at the doorAccess · up to 23 pts
Your edge answers every non-browser client with a block or interstitial. Web Bot Auth (HTTP Message Signatures, RFC 9421) lets an agent prove who it is per request; Cloudflare, Akamai, AWS WAF and Vercel can verify it. Allow verified agents and keep challenging the rest.
# Cloudflare: Security → Bots → Verified bots → allow category "AI Agent" # Or verify yourself: fetch the agent's key from # https://<signature-agent>/.well-known/http-message-signatures-directory # and check the Signature / Signature-Input headers (tag="web-bot-auth")
2Stop disallowing identified user-directed agents in robots.txtAccess · up to 14 pts
You disallow 16 agents that act live for a person. Crawler rules (GPTBot, ClaudeBot) can stay; user-directed agents are the ones a customer sends.
User-agent: GoogleAgent-Mariner User-agent: GoogleAgent-Shopping User-agent: Manus-User User-agent: MistralAI-User User-agent: Devin User-agent: Copilot Allow: /
3Give agents a front doorProtocols · up to 12 pts
A Universal Commerce Protocol manifest for shopping, or an MCP server card for everything else. An agent that can call an API never needs to hold a customer's password.
GET /.well-known/ucp
{ "ucp": { "version": "2026-01-23",
"services": { "dev.ucp.shopping": [{ "transport": "rest", "endpoint": "https://www.opentable.com/api/ucp/v1" }] },
"capabilities": { "dev.ucp.shopping.checkout": [ … ] } } }
# or
GET /.well-known/mcp.json
{ "name": "OpenTable", "endpoint": "https://www.opentable.com/mcp", "transport": "streamable-http" }4Publish llms.txtSurface · up to 8 pts
A short markdown map of what the site does, where the important pages are, and how to search or check out.
/llms.txt # OpenTable > One paragraph on what this site is for. - [Products](/products): how to search and filter - [Checkout](/checkout): steps and constraints
5Say your stance with a Content-SignalConsent · up to 3 pts
Two lines in robots.txt end the guessing game. ai-input covers agents reading pages on a person's behalf.
User-Agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: /