monperrus / crawler-user-agentsLinks
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome
β1,318Updated 2 months ago
Alternatives and similar repositories for crawler-user-agents
Users that are interested in crawler-user-agents are comparing it to the libraries listed below
Sorting:
- π€/π¨βπ¦° Detect bots/crawlers/spiders using the user agent stringβ1,102Updated 2 weeks ago
- Bot detection library that runs in the browser. Detects automation tools and frameworks. No server required, runs 100% on the client. MITβ¦β1,321Updated this week
- β589Updated 8 months ago
- Is headless chrome currently detectable? Let's pit the detections and detection evasions against eachother.β660Updated 4 years ago
- Daily updated repository for https://github.com/disposable/disposableβ560Updated this week
- A list of disposable email domainsβ1,354Updated 8 months ago
- A JavaScript library for generating random user agents with data that's updated daily.β1,127Updated last month
- reliable fake and temp email filter solution for site operatorsβ309Updated this week
- Cross-language temporary (disposable/throwaway) email detection library. Covers 55 734+ fake email providers.β1,807Updated 3 weeks ago
- Fingerprinting script of Fingerprint-Scannerβ257Updated 8 months ago
- An open source list of ASNs known to belong to cloud, managed hosting, and colo facilities.β437Updated 9 months ago
- Access https://infosimples.github.io/detect-headless to run several headless detection tests against your browser.β298Updated 5 years ago
- Device information and digital fingerprinting written in pure JavaScript.β2,207Updated 2 years ago
- Implementation of canvas fingerprinting based on the Picasso paperβ157Updated last year
- Luminati HTTP/HTTPS Proxy managerβ800Updated last month
- An always up-to-date list of the top 100 HTTP user-agents most used over the Internet.β295Updated last week
- JavaScript detection of Incognito & other private browsing modes on Chrome, Edge, Safari, Brave, Firefox, Opera and MSIE.β474Updated 2 months ago
- A list of temporary email providersβ1,151Updated 4 months ago
- Block bad, possibly even malicious web crawlers (automated bots) using Nginxβ862Updated 5 years ago
- A proxy back end for Google Tag Manager & Google Analyticsβ255Updated 7 months ago
- This repository contains instructions how to use the free IP Address API. The databases are: ASN database, Geolocation database, hosting β¦β112Updated this week
- Chrome Headless docker images built upon alpine official imageβ2,113Updated last week
- Whois server list for all top level domains.β325Updated 5 years ago
- Rotating TOR proxy with Dockerβ1,194Updated last year
- A precise user agent parser and device detector written in TypeScriptβ495Updated last year
- Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.β954Updated last week
- DFPM is a browser extension for detecting browser fingerprinting.β124Updated 2 years ago
- Get unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more.β2,586Updated last month
- JavaScript domain name parser based on the Public Suffix Listβ427Updated 7 months ago
- Easily create XML sitemaps for your website.β446Updated last year