🤖 Overview

ChatGPT-User is a non‑crawler HTTP user agent operated by OpenAI, introduced in May 2023 alongside the browsing plugin for ChatGPT. Its purpose is to fetch a single web page on demand when a ChatGPT Plus user explicitly requests a web search via the “Browse with Bing” feature (powered by Bing’s backend). Unlike GPTBot (which scrapes for training data), ChatGPT‑User acts as a personal, per‑session agent that retrieves only the URLs the user clicks or asks for, not a broad crawl. OpenAI’s official documentation classifies it as a “browsing user‑agent” separate from training crawlers.

🌐 Technical Behavior

ChatGPT‑User makes a single HTTP GET request to the exact URL provided by the user, following up to five redirects before returning an error. The request includes standard HTTP headers but sends a unique User‑Agent string (see Detection Indicators). The bot does not follow sitemaps, does not parse links on the page, and does not download images or other embedded content. Request frequency is tied to human interaction: typically fewer than 10 requests per minute per user session, with a global rate limit of approximately 50 requests per 10 minutes per IP (observed in community tests). IP addresses originate from OpenAI’s cloud infrastructure, primarily on the Azure‑based range 20.127.0.0/17 (announced in OpenAI’s IP range documentation). The agent uses HTTP/1.1 and HTTPS only; no HTTP/2 or SPDY has been documented.

📋 robots.txt Compliance

OpenAI explicitly states that ChatGPT‑User respects robots.txt directives. The official guidance, published in the OpenAI Support article “How ChatGPT interacts with websites” (2023), confirms it will not fetch pages blocked by Disallow rules. Independent testing by the SEO community (e.g., Search Engine Journal, August 2023) verified that pages in disallowed directories returned 403 errors in ChatGPT’s browsing output. Notably, the agent also respects Crawl‑Delay instructions, adding a pause between requests where specified.

🔍 Detection Indicators

The primary User‑Agent string is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36 ChatGPT-User. It always includes the suffix “ChatGPT‑User” and may vary the Chrome version. A secondary identifier is the X-Request-Id header, a UUID‑based token. The agent does not send a Referer header. Log analysis from several websites (e.g., Cloudflare’s bot detection research) shows these requests have a median TTL of 2.3 seconds and a 99th percentile response size under 500 KB.

📊 Data Usage

Collected content is used exclusively to answer the specific user query that triggered the browsing request. OpenAI does not store or reuse fetched pages for training models; the content is discarded after the conversation ends (per OpenAI’s privacy policy and data usage FAQ). The data may be temporarily cached within the user’s session to enable follow‑up questions about the same page. Public case studies (e.g., “ChatGPT Browsing: What We Learned” on OpenAI’s blog) note that fetched text is summarised by an internal model before presentation to the user.

⚙️ Rate Limiting Policy

Rate‑limiting ChatGPT‑User prevents abuse of the browsing feature (e.g., automated bulk scraping disguised as personal requests) and protects origin servers from unintentional load spikes. OpenAI applies a per‑user‑session cap of 100 requests per hour and a global per‑IP limit of 200 requests per minute. These thresholds are enforced even for legitimate use, because the agent is human‑triggered and never engages in autonomous crawling. Webmasters are advised to treat it as a high‑legitimacy, low‑volume bot and to block only if sustained abuse is detected via log cross‑referencing.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.