Scan a URL
The Index › Grocery & food › instacart.com
C

Instacartinstacart.com

56/ 100 · Agentic Usability Score
SelectiveScanned 2026-09-22 · cloudflare-edge · method v0.1Edge: cloudflare
Muse
Meta
?Untested
ChatGPT Agent
OpenAI
◐Partial
Claude
Anthropic
⊘Disallowed
Comet
Perplexity
⊘Disallowed
Grok Bot
xAI
⊘Disallowed
Gemini
Google
⊘Disallowed
Manus
Manus
⊘Disallowed
Instinct
Spear Street
?Untested

Score breakdown

Five pillars. Task completion is pending until the live task runner ships; the other four are rescaled to 100.
Accessrobots.txt stance + live door test
23/40
Consent & identity pathContent-Signal, published agent policy
7/10
Machine-readable surfacellms.txt, JSON-LD, landmarks, SSR
17/25
Agent protocolsUCP, MCP, A2A, OpenAPI
3/15
Task completionlive agent run: search → cart → checkout
pending

Evidence

Generated by the scanner from this run. Every line is reproducible with the open-source probe.
  • ✕robots.txt disallows 18 of 19 user-directed agents via a default Disallow: / for unlisted bots: Claude-User, Perplexity-User, GoogleAgent-Mariner, GoogleAgent-Shopping, Manus-User, MistralAI-User, Devin, Copilot…
  • ✓Homepage serves a normal page to ChatGPT-User, Claude-User, Perplexity-User and GoogleAgent-Mariner
  • ◐robots.txt restricts some paths for ChatGPT-User
  • ◐Also disallows 14 of 15 training/search crawlers (not scored; agents ≠ crawlers)
  • ◐No Content-Signal in robots.txt
  • ◐No llms.txt
  • ✓Homepage carries 1 JSON-LD block
  • ✓Semantic landmarks present (main, nav)
  • ✓ChatGPT plugin manifest at /.well-known/ai-plugin.json
Identified asHTTPResult
ChatGPT-User200normal page
Claude-User200normal page
Perplexity-User200normal page
GoogleAgent-Mariner200normal page
browser (human)200normal page

What Instacart would need to change

1Stop disallowing identified user-directed agents in robots.txtAccess · up to 16 pts

You disallow 18 agents that act live for a person. Crawler rules (GPTBot, ClaudeBot) can stay; user-directed agents are the ones a customer sends.

User-agent: Claude-User
User-agent: Perplexity-User
User-agent: GoogleAgent-Mariner
User-agent: GoogleAgent-Shopping
User-agent: Manus-User
User-agent: MistralAI-User
Allow: /

2Give agents a front doorProtocols · up to 12 pts

A Universal Commerce Protocol manifest for shopping, or an MCP server card for everything else. An agent that can call an API never needs to hold a customer's password.

GET /.well-known/ucp
{ "ucp": { "version": "2026-01-23",
  "services": { "dev.ucp.shopping": [{ "transport": "rest", "endpoint": "https://www.instacart.com/api/ucp/v1" }] },
  "capabilities": { "dev.ucp.shopping.checkout": [ … ] } } }

# or
GET /.well-known/mcp.json
{ "name": "Instacart", "endpoint": "https://www.instacart.com/mcp", "transport": "streamable-http" }

3Publish llms.txtSurface · up to 8 pts

A short markdown map of what the site does, where the important pages are, and how to search or check out.

/llms.txt
# Instacart
> One paragraph on what this site is for.
- [Products](/products): how to search and filter
- [Checkout](/checkout): steps and constraints

4Say your stance with a Content-SignalConsent · up to 3 pts

Two lines in robots.txt end the guessing game. ai-input covers agents reading pages on a person's behalf.

User-Agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /