monperrus / crawler-user-agentsLinks
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome
β1,298Updated last week
Alternatives and similar repositories for crawler-user-agents
Users that are interested in crawler-user-agents are comparing it to the libraries listed below
Sorting:
- π€/π¨βπ¦° Detect bots/crawlers/spiders using the user agent stringβ1,068Updated 2 weeks ago
- Daily updated repository for https://github.com/disposable/disposableβ530Updated this week
- Bot detection library that runs in the browser. Detects automation tools and frameworks. No server required, runs 100% on the client. MITβ¦β1,277Updated last week
- β579Updated 6 months ago
- Cross-language temporary (disposable/throwaway) email detection library. Covers 55 734+ fake email providers.β1,789Updated this week
- Is headless chrome currently detectable? Let's pit the detections and detection evasions against eachother.β660Updated 4 years ago
- Device information and digital fingerprinting written in pure JavaScript.β2,201Updated 2 years ago
- A list of temporary email providersβ1,144Updated last month
- A list of disposable/temporary email address domainsβ1,173Updated last month
- JavaScript detection of Incognito & other private browsing modes on Chrome, Edge, Safari, Brave, Firefox, Opera and MSIE.β457Updated 3 weeks ago
- An open source list of ASNs known to belong to cloud, managed hosting, and colo facilities.β423Updated 6 months ago
- Easily create XML sitemaps for your website.β439Updated last year
- Whois server list for all top level domains.β325Updated 5 years ago
- A JavaScript library for generating random user agents with data that's updated daily.β1,091Updated this week
- Fingerprinting script of Fingerprint-Scannerβ253Updated 6 months ago
- The world's easiest, smallest and powerful visitor identifier for browsers.β544Updated 2 weeks ago
- Detects ad blockers (AdBlock, ...)β1,907Updated last year
- Block bad, possibly even malicious web crawlers (automated bots) using Nginxβ862Updated 4 years ago
- reliable fake and temp email filter solution for site operatorsβ290Updated this week
- JavaScript domain name parser based on the Public Suffix Listβ422Updated 4 months ago
- Implementation of canvas fingerprinting based on the Picasso paperβ155Updated last year
- NGINX module for Brotli compressionβ2,183Updated last year
- List of major web + mobile browser user agent strings. +1 Bonus script to scrape :)β454Updated last year
- A proxy back end for Google Tag Manager & Google Analyticsβ255Updated 4 months ago
- Access https://infosimples.github.io/detect-headless to run several headless detection tests against your browser.β293Updated 5 years ago
- Automatic PageSpeed optimization module for Nginxβ4,353Updated 2 years ago
- Apache Block Bad Bots, (Referer) Spam Referrer Blocker, Vulnerability Scanners, Malware, Adware, Ransomware, Malicious Sites, Wordpress Tβ¦β915Updated this week
- Get unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more.β2,549Updated 2 weeks ago
- A precise user agent parser and device detector written in TypeScriptβ491Updated last year
- A free, open-source javascript fingerprinting libraryβ720Updated last week