metaeuro-web-search
MetaEuro Web Search is a legitimate web crawler operated by Meta Platforms, Inc. (formerly Facebook) for indexing publicly accessible web content to power its European-focused web search product and to improve AI models used within Meta’s ecosystem. Officially documented in Meta’s crawler registry, this bot is distinct from Meta’s general-purpose crawlers and is specifically deployed to comply with European data regulations while supporting Meta’s search features on platforms like Facebook and Instagram.
The crawler sends HTTP requests with a standard user-agent string and employs both HTTP/1.1 and HTTP/2 protocols. According to Meta’s published infrastructure, it originates from IP ranges within Meta’s autonomous system AS32934, including subnets such as 69.171.224.0/19 and 31.13.24.0/21. It typically issues requests at a steady rate of 10–20 per second, though burst activity may occur during initial index updates. The crawler respects Crawl-Delay directives in robots.txt and supports conditional If-Modified-Since headers to reduce bandwidth usage. Reverse DNS lookups of its IP addresses resolve to hostnames under *.fbsv.com or *.facebook.com.
Meta explicitly states in its official crawler documentation that MetaEuro Web Search honors all Disallow directives in robots.txt. The crawler’s behavior aligns with Meta’s published policy, which includes support for User-agent: MetaExternalAgent and User-agent: Meta-ExternalAgent lines. Failure to comply would contradict Meta’s own guidelines and has not been reported in any security advisories.
The primary User-Agent string is MetaExternalAgent/1.0 with additional variants like Meta-ExternalAgent/1.0 and MetaEuro/1.0. Behavioral fingerprints include consistent intervals between requests (typically 0.5–2 seconds) and a preference for crawled pages over non‑text content. The crawler does not alter Accept-Language headers unusually and sends standard User-Agent and From fields containing Meta’s contact information.
Collected data feeds into Meta’s internal web search index, used to surface relevant web results within Facebook’s search bar and Instagram’s explore features. Additionally, a subset of crawled content is used to train Meta’s large language models (such as Llama) and improve content recommendation algorithms, as noted in Meta’s AI research publications and their Data Policy. No personally identifiable information is intentionally collected.
While MetaEuro Web Search is not malicious, its sustained crawl rate can overwhelm smaller sites if left unchecked. Therefore, rate-limiting with a threshold of 20–30 requests per second per IP is recommended to prevent resource exhaustion while still allowing legitimate indexing. Meta also advises webmasters to use Crawl-Delay in robots.txt to control pace.
— Imperva Bad Bot Report 2026
How much of your traffic is automated? Get your personal bot traffic report and see exactly what's hitting your server — completely free.
📊 Get My Bot ReportSign up in seconds · No card required
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
Stay up to date with the latest from Boteraser.
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Manage your cookie preferences below:
Essential cookies enable basic functions and are necessary for the proper function of the website.
CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.
Service URL: developers.cloudflare.com (opens in a new window)
These cookies are needed for adding comments on this website.
These cookies are used for managing login functionality on this website.
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Service URL: policies.google.com (opens in a new window)
You can find more information in our Cookie Policy and Privacy Policy.