monperrus / crawler-user-agents
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome
β1,272Updated this week
Alternatives and similar repositories for crawler-user-agents
Users that are interested in crawler-user-agents are comparing it to the libraries listed below
Sorting:
- π€/π¨βπ¦° Detect bots/crawlers/spiders using the user agent stringβ1,021Updated this week
- A list of disposable email domainsβ1,340Updated 2 months ago
- Whois server list for all top level domains.β321Updated 4 years ago
- A list of disposable/temporary email address domainsβ1,116Updated 2 weeks ago
- Allows you to detect the extension AdBlock (and other)β622Updated 6 years ago
- A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and conβ¦β381Updated 2 years ago
- An open source list of ASNs known to belong to cloud, managed hosting, and colo facilities.β391Updated 3 months ago
- Daily updated repository for https://github.com/disposable/disposableβ505Updated this week
- Learn how identifiable you are on the Internetβ696Updated 4 years ago
- The ultimate guide on preventing Website Scrapingβ1,500Updated last year
- The main project repositoryβ436Updated this week
- Can I emailβ¦ Support tables for HTML and CSS in emails.β860Updated 3 weeks ago
- Validate_email verify if an email address is valid and really existsβ582Updated 2 years ago
- This is a project for a browser fingerprinting technique that can track users not only within a single browser but also across different β¦β1,270Updated 3 years ago
- Wouldn't it be nice if `User-Agent` was a (set of) client hints?β603Updated last week
- Some ideas around spoofing and detecting user agents.β254Updated 9 years ago
- Bot detection library that runs in the browser. Detects automation tools and frameworks. No server required, runs 100% on the client. MITβ¦β1,215Updated 3 months ago
- JavaScript domain name parser based on the Public Suffix Listβ406Updated last month
- Privacy-focused encrypted email for everyone. All-in-one alternative to Gmail + Mailchimp + Sendgrid.β1,124Updated this week
- Easily create XML sitemaps for your website.β432Updated 10 months ago
- A proxy back end for Google Tag Manager & Google Analyticsβ251Updated last month
- reliable fake and temp email filter solution for site operatorsβ260Updated this week
- Rotating TOR proxy with Dockerβ1,172Updated last year
- Distributed crawler powered by Headless Chromeβ5,574Updated 2 years ago
- Browser sniffing gone too far β A useragent parser library for PHPβ1,825Updated 11 months ago
- Cross-language temporary (disposable/throwaway) email detection library. Covers 55 734+ fake email providers.β1,748Updated this week
- πA small webpage library to control the execution of (third party) scriptsβ800Updated 8 months ago
- A list of temporary email providersβ1,115Updated last week
- Sendwithus Open Source Email Templatesβ1,820Updated last year
- A Fast and Free Email Verification API written in Goβ1,048Updated 6 years ago