🤖 Overview

Toweyabot is a legitimate web crawler operated by Toweya Ltd, a UK-based company that provides website monitoring, SEO analysis, and uptime tracking services. First documented in 2018, the bot collects publicly accessible data to feed into the Toweya dashboard, which offers performance metrics, broken link detection, and page-speed evaluations for webmasters and agencies. According to Toweya’s official documentation at https://www.toweya.com/bot, its sole purpose is to scan websites for technical health and search-engine optimization indicators, not for AI training or content aggregation.

🌐 Technical Behavior

Toweyabot typically crawls at a moderate rate of one request per second per domain, as stated in Toweya’s rate-limiting guidelines. It uses HTTP/1.1 and supports both IPv4 and IPv6, with IP ranges published in Toweya’s documentation and in public SPF records (e.g., 185.151.32.0/22 and 2a00:5760:2::/48). The bot sends a Referer header set to “https://www.toweya.com/” and includes a From header with the contact address [email protected]. It respects Cache-Control and ETag headers to minimise server load and avoids crawling pages that return 404 or 429 status codes.

📋 robots.txt Compliance

Toweya explicitly states that Toweyabot respects the robots.txt standard and honors Disallow directives for both its main crawler and any sub‑bots. The official documentation at Toweya.com/bot confirms that the bot checks robots.txt before every crawl and immediately backs off if a disallowed path is encountered. Web administrators can also use a Crawl‑Delay directive to further throttle the bot’s request speed.

🔍 Detection Indicators

The primary User-Agent string is “Toweyabot” (case‑sensitive), sometimes followed by a version number such as “Toweyabot/1.0”. Some logs show an additional comment field like “(compatible; Toweyabot; +https://www.toweya.com/bot)”. The bot also includes a custom HTTP header X‑Toweya‑Bot set to “true”. Behavioral fingerprints include a fixed crawl interval of approximately one second and the absence of JavaScript execution or cookie storage.

📊 Data Usage

Collected data is used exclusively for Toweya’s website monitoring dashboard, which tracks page load times, HTTP status codes, SSL certificate validity, and SEO‑related meta tags. No data is sold, shared, or used for AI model training. Toweya’s privacy policy at https://www.toweya.com/privacy confirms that raw crawl logs are retained for 30 days only and are anonymised before storage.

⚙️ Rate Limiting Policy

Although Toweyabot conducts less than one request per second under normal conditions, webmasters may choose to rate‑limit the bot to protect server resources from cumulative load across multiple concurrent scans. The policy rationale is that threshold‑based blocking (e.g., via mod_evasive or fail2ban) should be applied only if the bot exceeds five requests per second, as documented in Toweya’s support pages.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.