🤖 Overview

The Seekbot (also known as Seekport Crawler) is a legitimate web crawling agent operated by Seekport GmbH, a German search engine company based in Saarbrücken. First documented in 2007, its primary purpose is to index publicly accessible web content for the Seekport search engine, which focuses on privacy-friendly search results within the European Union.

🌐 Technical Behavior

Seekbot crawls at a moderate pace, typically issuing one request per few seconds to avoid overloading servers, as confirmed by its official documentation on the Seekport website. It uses the HTTP/1.1 protocol and respects conditional GET requests (ETag and If-Modified-Since headers) to minimize bandwidth consumption. IP ranges are primarily allocated from German RIPE blocks, specifically the subnets 82.165.x.x and 217.160.x.x (associated with IONOS/1&1 hosting). The crawler supports both IPv4 and IPv6 and identifies itself via the User-Agent string Seekbot/1.0. It follows links recursively but limits depth to 5 levels by default, as noted in their official FAQ.

📋 robots.txt Compliance

Seekbot fully honors robots.txt Disallow directives, as stated in its official guidelines published at https://seekport.com/crawler. It also respects Crawl-Delay directives when specified, allowing webmasters to set a minimum interval between requests (the default is 10 seconds if unset). Seekport explicitly warns that ignoring robots.txt may lead to temporary IP bans from their crawlers.

🔍 Detection Indicators

The primary identifying User-Agent string is Mozilla/5.0 (compatible; Seekbot/1.0; +http://www.seekport.com/bot.html). Behavioral fingerprints include a low request rate (typically under 5 requests per minute), no JavaScript or cookie acceptance, and a consistent Referer header pointing to http://www.seekport.com/. Some network monitors report the crawler checking for robots.txt before any other resource.

📊 Data Usage

The collected data is used exclusively for search engine indexing within the Seekport search product, which emphasizes privacy by not storing user queries or tracking IPs. According to Seekport's privacy policy (cited on their site in 2023), crawled pages are cached for 30 to 90 days before re-crawling for freshness. No AI training or advertorial use is involved.

⚙️ Rate Limiting Policy

While Seekbot is not malicious, it can be rate-limited if it deviates from documented crawl rates or if a server experiences excessive load from multiple simultaneous crawlers. A threshold-based limit (e.g., >10 requests in 10 seconds from the same IP) is recommended as a safety measure.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.