🤖 Overview

LinerBot is a legitimate web crawler operated by Liner Co., Ltd., the South Korean startup behind the Liner application (liner.ai). First publicly documented in 2020, its primary purpose is to index publicly accessible web content to support Liner’s core features: highlighter, note‑taking, and AI‑powered summarization and question‑answering. The collected data feeds into Liner’s proprietary search engine and machine learning models, which enhance user productivity by extracting key information from articles and documents.

🌐 Technical Behavior

LinerBot sends requests over HTTP/1.1 with a default crawl frequency of approximately one request every 2–3 seconds, though the operator may adjust this based on server load. Official documentation (available at liner.ai/crawler) states it prioritizes textual content and avoids binary files such as images, PDFs, or videos unless explicitly linked as HTML. The crawler originates from a known set of IPv4 addresses registered under Liner’s ASN (AS396943 as of April 2025), which are shared with the company’s other infrastructure. It respects Cache‑Control and If‑Modified‑Since headers to minimise redundant downloads. The user‑agent token is always accompanied by a contact email () in the HTTP From header, allowing webmasters to reach the operator directly.

📋 robots.txt Compliance

Liner’s official robot exclusion policy, published on their website, confirms that LinerBot fully honours robots.txt Disallow directives (RFC 9309). The crawler checks the file before each crawl session and immediately stops accessing any URL path listed with Disallow. No evidence of intentional or accidental violation has been reported in public security advisories or webmaster forums as of 2025.

🔍 Detection Indicators

The primary User‑Agent string is LinerBot/1.0 (+https://liner.ai/crawler). Secondary variants may append a build number (e.g., LinerBot/2.0). The crawler also sets the HTTP header From: [email protected] and includes a Referer header pointing to the original page when following internal links. Behavioural fingerprint: the crawler almost never requests JavaScript or CSS files and does not send cookies unless a session requires it (rare).

📊 Data Usage

Data collected by LinerBot is used exclusively for Liner’s internal index and AI model training—specifically for improving the accuracy of their paragraph‑level highlighting, automatic summarisation, and the “Ask AI” feature that answers questions based on user‑saved content. The company states in its privacy policy (liner.ai/privacy) that crawled text is not sold or shared with third‑party advertisers and is stored in encrypted form.

⚙️ Rate Limiting Policy

Because LinerBot can scale its crawl rate dynamically when ingesting large sites, webmasters are advised to implement rate limiting (e.g., 10–20 requests per minute) to prevent resource exhaustion. The rationale is defensive: while the bot is benign, aggressive crawling of pages with heavy database queries could degrade site performance, justifying threshold‑based blocking for non‑compliant crawlers.

ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.