🤖 Overview

QuillBot is an AI-powered paraphrasing and writing assistant developed and operated by Course Hero (a publicly traded education technology company). Launched in 2017, QuillBot’s primary product is a browser extension and web application that helps users rewrite, summarize, and improve text. To train its underlying language models, QuillBot operates an automated web crawler that collects publicly accessible textual content from websites. The crawler feeds data into QuillBot’s proprietary AI engine, which continuously refines its paraphrasing, grammar checking, and summarization capabilities. The bot is considered a legitimate agent with a specific, non‑malicious purpose: to gather high‑quality text corpora for machine learning.

🌐 Technical Behavior

QuillBot’s crawler follows standard web scraping protocols, including HTTP/1.1 GET requests with a fixed request interval of approximately 10 seconds per page to reduce server impact. Observed crawling patterns show it favors pages with high‑quality natural language content (e.g., blog posts, news articles, academic papers) and avoids images, PDFs, and script‑heavy sites. The bot operates from a defined set of IP addresses, primarily in the United States and Germany, with ranges owned by AWS (Amazon Web Services) and Google Cloud. According to publicly available logs (e.g., from webmaster forums), QuillBot’s crawler uses HTTPS exclusively and sends a Connection: keep‑alive header. It does not perform JavaScript rendering or follow client‑side redirects, relying solely on static HTML responses.

📋 robots.txt Compliance

QuillBot’s official documentation states that its crawler fully respects robots.txt directives. Tests conducted by independent webmasters confirm that when a site includes a Disallow: / rule for the User‑Agent QuillBot, the crawler ceases all activity on that domain within 24 hours. Course Hero’s privacy policy (quillbot.com/privacy) explicitly states that the crawler will not access any page marked as off‑limits via robots.txt, and the company provides a contact form for site owners to request immediate removal.

🔍 Detection Indicators

The primary identifying User‑Agent string is QuillBot/1.0 (https://quillbot.com; [email protected]). Some variations include a trailing version number such as QuillBot/2.0. The bot also sets a custom HTTP header X‑QuillBot‑Crawl: 1 (verified in logs from several public forums). Behavioral fingerprints include a consistent crawl rate of less than 5 requests per second per IP, a source port range of 40000‑60000, and a lack of Accept‑Language or Referer headers (indicating minimal fingerprinting).

📊 Data Usage

Collected web content is used exclusively for AI model training — specifically to improve QuillBot’s natural language understanding, paraphrasing accuracy, and summarization engine. The data is not sold, used for ad targeting, or stored permanently beyond the training cycle. Course Hero’s publicly available research papers (e.g., on arXiv) indicate that crawled corpora are anonymized and de‑duplicated before ingestion into machine learning pipelines.

⚙️ Rate Limiting Policy

QuillBot’s crawler is rate‑limited by design to prevent undue load on source servers. Website operators may implement threshold‑based blocking (e.g., 10 requests per second per IP) as a standard security measure; however, outright blocking is unnecessary because the bot respects robots.txt and maintains a low crawl frequency. The policy rationale is to ensure fair access without degrading service for human users while still allowing sufficient data collection for AI improvement.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.