monperrus / crawler-user-agents
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome
β1,253Updated this week
Alternatives and similar repositories for crawler-user-agents:
Users that are interested in crawler-user-agents are comparing it to the libraries listed below
- π€/π¨βπ¦° Detect bots/crawlers/spiders using the user agent stringβ998Updated this week
- Bot detection library that runs in the browser. Detects automation tools and frameworks. No server required, runs 100% on the client. MITβ¦β1,183Updated last month
- Fingerprinting script of Fingerprint-Scannerβ245Updated 3 weeks ago
- β561Updated 3 weeks ago
- Device information and digital fingerprinting written in pure JavaScript.β2,148Updated 2 years ago
- Creepy device and browser fingerprintingβ1,764Updated 3 weeks ago
- A list of disposable email domainsβ1,332Updated last week
- Is headless chrome currently detectable? Let's pit the detections and detection evasions against eachother.β654Updated 3 years ago
- An open source list of ASNs known to belong to cloud, managed hosting, and colo facilities.β379Updated last month
- β138Updated last month
- Daily updated repository for https://github.com/disposable/disposableβ491Updated this week
- Splits a hostname into subdomains, domain and (effective) top-level domains.β511Updated 2 months ago
- Javascript scraping module based on puppeteer for many different search engines...β557Updated 2 years ago
- The world's easiest, smallest and powerful visitor identifier for browsers.β459Updated 11 months ago
- High performance JSON IP and GeoIP REST API (IP Geolocation)β867Updated last year
- Nginx GeoIP2 moduleβ1,022Updated 11 months ago
- Block bad, possibly even malicious web crawlers (automated bots) using Nginxβ865Updated 4 years ago
- End user oriented web performance testing and beaconingβ1,880Updated 5 months ago
- A precise user agent parser and device detector written in TypeScriptβ483Updated last year
- An always up-to-date list of the top 100 HTTP user-agents most used over the Internet.β240Updated last week
- A platform detection library.β3,233Updated last year
- Worldβs single largest Internet domains datasetβ753Updated 3 weeks ago
- This is a project for a browser fingerprinting technique that can track users not only within a single browser but also across different β¦β1,268Updated 3 years ago
- A list of disposable/temporary email address domainsβ1,071Updated 2 weeks ago
- Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with antiβ¦β4,268Updated this week
- web fingerprinting researchβ218Updated last year
- reliable fake and temp email filter solution for site operatorsβ247Updated this week
- How to detect puppeteer with 100% accuracyβ108Updated 3 years ago
- A database of free and disposable email domains and a handy Node.js module for querying it.β352Updated 11 months ago
- Comprehensive country code information, including ISO 3166 codes, ITU dialing codes, ISO 4217 currency codes, and many othersβ926Updated 3 weeks ago