🤖 Overview

Teleport is a web crawler operated by Teleport Search (teleportsearch.com), a privacy-focused search engine launched in 2020. Its primary purpose is to index publicly accessible web pages for inclusion in Teleport’s search results, enabling users to find relevant content without tracking. The bot is officially documented at teleportsearch.com/crawler and is registered in the Robots Exclusion Protocol standard.

🌐 Technical Behavior

The crawler employs HTTP/1.1 and HTTP/2 protocols, sending GET requests with the user-agent string TeleportBot/1.0. IP addresses originate from the range 104.28.0.0/14, part of Cloudflare’s infrastructure, though Teleport Search manages its own crawl orchestrators. The typical crawl rate is one request per two seconds per domain, with a maximum of 60 requests per minute. It respects Crawl-Delay directives and reduces frequency on server errors. The bot follows links recursively up to depth five and includes Accept-Encoding: gzip, deflate headers. Conditional requests with If-Modified-Since and If-None-Match are supported to avoid re-downloading unchanged content. The From header is set to [email protected] for easy identification.

📋 robots.txt Compliance

Teleport fully adheres to the Robots Exclusion Protocol as documented in its official guidelines. It honors all Disallow directives and supports the Crawl-Delay field. Webmasters can block the crawler entirely by adding User-Agent: TeleportBot with Disallow: /. The bot also respects Allow directives for granular control.

🔍 Detection Indicators

The primary indicator is the User-Agent string TeleportBot/1.0 (case-sensitive). Additionally, the From header with value [email protected] confirms identity. Reverse DNS lookups on crawling IPs show hostnames ending in .crawl.teleportsearch.com. No other behavioral fingerprints are required for identification.

📊 Data Usage

Collected data is used exclusively for building and updating Teleport Search’s web index. The company explicitly states it does not use crawled content for AI training, advertising, or resale. Anonymized metrics about crawl frequency and index size may be published, but no personal data is stored. The search engine is funded through privacy-respecting means rather than data monetization.

⚙️ Rate Limiting Policy

Teleport is rate-limited because unrestricted crawling could overwhelm smaller sites. A recommended policy is to allow up to 5 requests per second per IP with a burst of 10, ensuring timely indexing while protecting server stability. This threshold aligns with standard practices for legitimate crawlers.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.