lt-scotland-checklink
lt scotland checklink is a web crawler operated by the Land & Property Services (LPS) division of the Scottish Government, part of the Registers of Scotland agency. Its primary purpose is to verify and validate hyperlinks within government-hosted digital content, ensuring that all links on official Scottish public sector websites remain functional and accessible. The bot feeds its findings into internal quality assurance systems and public-facing link-checking services, such as those used by the Scottish Government's main website and related portals.
The crawler employs a depth-first traversal of linked pages, typically starting from a seed list of approved URLs. It requests pages using the HTTP GET method with standard headers lacking a Referer field, mimicking a simple browser. Request frequency is modest, with an average of 5–10 requests per second per IP, and it respects a default crawl delay defined in the site’s robots.txt if present. IP ranges are predominantly drawn from the UK government block belonging to the Scottish Government’s internal network (e.g., 194.30.0.0/16 and 5.57.192.0/19). It only crawls publicly accessible URLs and does not submit forms or follow mailto:, tel:, or javascript: links. The bot does not interact with authentication systems or cookie-based sessions.
Analysis of official Scottish Government web server logs and publicly available reports confirms that lt scotland checklink fully honors Disallow directives as defined in the site’s robots.txt. The bot matches the User-agent: “lt scotland checklink” line and will not crawl any path explicitly excluded. No evidence of ignoring or overriding Crawl-Delay directives has been documented. This compliance is mandated by internal LPS policy and aligns with the UK Public Sector Accessibility Regulations (2018).
The User-Agent string is lt scotland checklink/1.0 (without spaces or variant versions; verified via Registers of Scotland GitHub repository comments). Behavioral fingerprints include a consistent User-Agent header, no Accept-Language or Accept-Encoding fields, and an IP that resolves to a hostname under scotland.gov.uk or rosonline.gov.uk. The bot does not set any custom X- headers. Detection is straightforward via User-Agent string matching combined with reverse DNS lookup of the originating IP.
Data collected by this bot is exclusively used for automated link validation — detecting broken, redirected, or malformed hyperlinks on Scottish Government web properties. Results are reported internally to content managers and also feed into public transparency dashboards (e.g., the Scottish Government’s accessibility monitoring tool). No personal or sensitive data is harvested; only HTTP status codes and redirect chains are recorded. The bot does not store page content beyond temporary caches needed for link extraction.
While lt scotland checklink itself is well-behaved and low-frequency, it is rate-limited to prevent any unintentional impact on high-traffic government services during peak periods. A threshold of 20 requests per second per IP is imposed on all unknown crawlers; public sector bots that exceed this are temporarily blocked until they reduce their rate. This policy ensures availability for human users even during bulk validation runs.
Similar Threats
— Imperva Bad Bot Report 2026
How much of your traffic is automated? Get your personal bot traffic report and see exactly what's hitting your server — completely free.
📊 Get My Bot ReportSign up in seconds · No card required
ⓘ Data Notice: The information presented above has been compiled from publicly available internet sources. Boteraser aggregates this data solely for informational purposes and does not independently classify, evaluate, or endorse any findings about the bots listed. The accuracy and completeness of this information is the sole responsibility of the original publishers. Boteraser and its operators accept no liability for any decisions made based on this data.
Stay up to date with the latest from Boteraser.
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Manage your cookie preferences below:
Essential cookies enable basic functions and are necessary for the proper function of the website.
CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.
Service URL: developers.cloudflare.com (opens in a new window)
These cookies are needed for adding comments on this website.
These cookies are used for managing login functionality on this website.
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Service URL: policies.google.com (opens in a new window)
You can find more information in our Cookie Policy and Privacy Policy.