monperrus / crawler-user-agents
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome
☆1,267Updated this week
Alternatives and similar repositories for crawler-user-agents:
Users that are interested in crawler-user-agents are comparing it to the libraries listed below
- Daily updated repository for https://github.com/disposable/disposable☆498Updated this week
- A list of disposable email domains☆1,339Updated last month
- IP Intelligence is a free Proxy VPN TOR and Bad IP detection tool to prevent Fraud, stolen content, and malicious users. Block proxies, V…☆321Updated 2 months ago
- a list of disposable email domains☆3,627Updated this week
- reliable fake and temp email filter solution for site operators☆255Updated this week
- A list of disposable/temporary email address domains☆1,095Updated 2 weeks ago
- Block bad, possibly even malicious web crawlers (automated bots) using Nginx☆865Updated 4 years ago
- NGINX module for Brotli compression☆2,139Updated 10 months ago
- Inline css into an html file.☆434Updated 3 weeks ago
- An open source list of ASNs known to belong to cloud, managed hosting, and colo facilities.☆384Updated 2 months ago
- Templates for Cloudflare Workers☆1,252Updated this week
- Apache Block Bad Bots, (Referer) Spam Referrer Blocker, Vulnerability Scanners, Malware, Adware, Ransomware, Malicious Sites, Wordpress T…☆872Updated this week
- A list of temporary email providers☆1,108Updated 2 months ago
- A list of role-based email addresses, used to detect whether an email address (e.g. team@domain.com) is likely not an individual☆73Updated last month
- Apache module for rewriting web pages to reduce latency and bandwidth.☆694Updated 2 years ago
- Whois server list for all top level domains.☆321Updated 4 years ago
- Allows you to detect the extension AdBlock (and other)☆622Updated 5 years ago
- Daily updated list of IP addresses / CIDR blocks used by data centers, cloud service providers, servers, etc.☆179Updated last week
- 🔐A small webpage library to control the execution of (third party) scripts☆800Updated 7 months ago
- A free solution to the EU, GDPR, and California Cookie Laws☆3,462Updated 10 months ago
- An easy way to check if users visiting your websites are bot or not.☆92Updated 7 years ago
- A node module for Google's Universal Analytics and Measurement Protocol☆964Updated 2 years ago
- Device detection (phone, tablet, desktop, mobile grade, os, versions)☆4,137Updated 4 years ago
- A database of free and disposable email domains and a handy Node.js module for querying it.☆353Updated 11 months ago
- Full list of timezones☆820Updated 11 months ago
- Library for extracting marketing attribution data from referrer URLs☆363Updated last month
- i18n for ISO 3166-1 country codes☆837Updated 2 months ago
- 🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent☆2,066Updated last month
- Scrape websites for text by CSS selector.☆628Updated last year
- Repository archived and moved to monorepo☆2,456Updated 4 months ago