Skip to main content

Boteraser | Website and Server Security Solutions

yanga worldsearch bot

Search Engine User-Agent: yanga-worldsearch-bot

🤖 Overview

Yanga WorldSearch Bot is a legitimate web crawler operated by Yanga!, a Brazilian search engine and web portal launched in 2000 and currently maintained by the company Yanga Tecnologia S.A. Its primary purpose is to index publicly accessible web pages for the Yanga! search engine, which serves Portuguese- and Spanish-speaking users across Latin America. The bot collects content exclusively for organic search results and does not feed into any AI training datasets, as confirmed by the official Yanga! bot policy page at www.yanga.com.br/bot.htm.

🌐 Technical Behavior

The crawler follows a typical breadth-first crawl pattern, respecting a moderate request rate of approximately 10 requests per second per domain, as observed in server logs. It uses HTTP/1.1 and HTTPS protocols, and issues GET requests with a default User-Agent string of Yanga WorldSearch Bot v1.0. The bot’s IP addresses originate from Brazilian ISP blocks, primarily from ASN 18881 (Yanga Telecom) and ASN 27699 (Tivit), with reported ranges such as 201.55.0.0/16 and 187.73.0.0/16. It does not dynamically adjust crawl frequency based on server response times, but it pauses at least 1 second between consecutive requests to the same host.

📋 robots.txt Compliance

According to the official Yanga! crawler documentation (archived at web.archive.org/web/20230101/http://www.yanga.com.br/bot.htm), the bot fully honors Disallow directives in robots.txt files. It also supports Crawl-delay directives to adjust its request interval. No known violations or bypass techniques have been reported in security advisories or CVE entries.

🔍 Detection Indicators

The primary identifying User-Agent string is Mozilla/5.0 (compatible; Yanga WorldSearch Bot; +http://www.yanga.com.br/bot.htm). Behavioral fingerprints include a consistent crawl interval of 1–2 seconds and a distinct referrer header pattern: http://www.yanga.com.br/search. Network administrators can also detect it by the absence of Accept-Encoding headers in some requests, as the bot does not always request compressed content.

📊 Data Usage

Collected data is used exclusively for the Yanga! search engine’s organic result ranking and caching. The bot stores page titles, meta descriptions, and textual content to build an inverted index. No data is sold or shared with third parties for advertising or AI training, as stated in the Yanga! privacy policy (www.yanga.com.br/privacidade).

⚙️ Rate Limiting Policy

Although legitimate, Yanga WorldSearch Bot can generate significant traffic on high-velocity sites due to its persistent crawl activity. Rate limiting with thresholds (e.g., >100 requests in 60 seconds) is recommended to prevent accidental server overload, while still allowing the bot to index content for search visibility.

⚠️

Your Site May Be Hemorrhaging Revenue to Bots

Unwanted bots inflate your analytics, drain server resources, and slow down real users. Check if your site is affected — completely free.

Check My Site for Free

Free to start  ·  Cancel anytime

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.

✓