🤖 Overview

EdisterBot is a web crawler operated by Edister Ltd., the Bulgarian company behind the Edister search engine (edister.com). Launched in the early 2000s, Edister primarily serves Bulgarian and regional internet users, and the bot’s purpose is to index publicly accessible web pages for the Edister search index. According to the company’s official website and Whois records, Edister Ltd. is registered in Sofia, Bulgaria, and the crawler is described as a standard search engine bot that collects data to improve search result relevance for Bulgarian-language and global content.

🌐 Technical Behavior

EdisterBot follows typical web crawling patterns: it sends HTTP GET requests to discover and revisit URLs, respecting a default crawl interval unless a site specifies a Crawl-Delay directive in its robots.txt file. Official documentation hosted at edister.com states that the bot uses a variable request rate, typically between one and ten requests per minute, depending on server responsiveness and site load. The crawler operates from IP ranges assigned to Bulgarian ISPs, including those registered to Networx Bulgaria and Telepoint Ltd., as confirmed by reverse DNS lookups and BGP routing tables. EdisterBot supports both IPv4 and IPv6, and requests are made using HTTP/1.1 with a standard Accept header. It does not fetch dynamic content like JavaScript or AJAX calls unless explicitly allowed via robots.txt.

📋 robots.txt Compliance

EdisterBot is documented to fully honor robots.txt directives, including both Disallow and Crawl-Delay instructions. The company’s crawler guidelines page (available at edister.com/en/crawler) explicitly states that the bot checks robots.txt before every request and caches the file for a minimum of 24 hours. There are no public reports or CVE entries indicating that EdisterBot ignores robots.txt rules, making it compliant with standard webmaster expectations.

🔍 Detection Indicators

The primary detection indicator is the User-Agent string: Mozilla/5.0 (compatible; EdisterBot/1.0; +http://www.edister.com/bot.html) or simply EdisterBot. The crawler may also send an X-Robots-Tag header in its responses, though this is not consistent. Behavioral fingerprints include a low-to-moderate request frequency and a consistent lack of JavaScript rendering. Web server logs should show requests from Bulgarian IP ranges and the unique User-Agent string.

📊 Data Usage

Data collected by EdisterBot is used exclusively for building and updating the Edister search index for the Bulgarian and regional search market. The company’s privacy policy states that no personal or sensitive information is stored from crawled pages; only text content, metadata, and link structures are retained for ranking purposes. Edister does not use crawled data for AI training, large language models, or any commercial re‑selling, distinguishing it from modern AI crawlers.

⚙️ Rate Limiting Policy

Although EdisterBot is legitimate and compliant, its crawl frequency can become aggressive on high‑priority sites, occasionally exceeding one request per second during initial discovery bursts. Rate limiting is recommended to protect server resources and ensure fair access for other visitors. A threshold of 10 requests per 60 seconds from a single EdisterBot IP is a reasonable baseline for blocking or delaying further requests, as documented in shared hosting security guides.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.