🤖 Overview

Bot Apoena is a web crawler operated by Apoena Tecnologia, a Brazilian company launched in 2018 to index publicly accessible web content for a proprietary search engine and business intelligence platform. It aggregates data for market research, competitive analysis, and trend detection services offered to enterprise clients across Latin America.

🌐 Technical Behavior

The crawler uses a headless Chromium engine to render JavaScript and capture dynamically generated pages. It sends requests at a default rate of 1-2 per second per domain, with configurable crawl delays via the Crawl-Delay directive in robots.txt. Its IP ranges are predominantly Brazilian, including blocks 177.54.0.0/16 and 191.242.0.0/16, as listed in the official IP documentation at https://apoena.com.br/ip-ranges. The bot supports both HTTP/1.1 and HTTPS, with a preference for HTTP/2 when available, and uses TLS 1.2 or higher. It follows a breadth-first crawling strategy, prioritizing new or updated content before recrawling previously indexed pages.

📋 robots.txt Compliance

According to Apoena’s official bot FAQ at https://apoena.com.br/bot, the crawler fully honors Disallow and Allow directives in robots.txt. It also respects X-Robots-Tag HTTP headers and meta robots HTML tags on individual pages. The company provides a feedback mechanism for webmasters to report any non-compliance issues, and they commit to addressing them within 48 hours.

🔍 Detection Indicators

The primary User-Agent string is Mozilla/5.0 (compatible; ApoenaBot/2.0; +https://apoena.com.br/bot-info). Alternate versions include Apoena-crawler/1.0 and Apoena-Scrape/3.0. The bot sets a custom HTTP header X-Apoena-Crawl: yes and uses Accept-Language pt-BR,pt;q=0.9. Its requests typically include a Referer header pointing to https://apoena.com.br.

📊 Data Usage

Collected data is used exclusively for search indexing within Apoena’s search engine and for training proprietary transformer-based NLP models focused on Portuguese language understanding, sentiment analysis, and trend detection. The company asserts that no personally identifiable information is retained beyond what is publicly available on the web, and they offer an opt-out mechanism via their bot management portal at https://apoena.com.br/opt-out.

⚙️ Rate Limiting Policy

Because the bot can send rapid sequential requests that may degrade server performance for other visitors, webmasters are advised to implement rate limiting with thresholds of 5 requests per second per IP. Apoena’s own documentation recommends this practice to balance legitimate indexing with server stability, emphasizing that it is a cooperative measure rather than a blocking policy.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.