chrisakroyd / robots-txt-parserLinks
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
☆15Updated 2 years ago
Alternatives and similar repositories for robots-txt-parser
Users that are interested in robots-txt-parser are comparing it to the libraries listed below
Sorting:
- NodeJS robots.txt parser with support for wildcard (*) matching.☆165Updated 2 months ago
- A module for node.js and the browser that takes in text and strips it of stopwords☆261Updated 3 weeks ago
- A Base32-Crockford encoded API Key generator and converter to turn UUIDs into human readable API Keys☆60Updated 3 years ago
- 🇫🇷 NodeJS language detection library using n-gram☆412Updated 5 years ago
- Very basic encrypt and decrypt node module☆270Updated 4 months ago
- Small tool to wait that all xhr are finished in puppeteer☆278Updated 3 weeks ago
- PHP Serialize/Unserialize in Javascript☆119Updated 10 months ago
- A custom rule bundle for @microlinkhq/metascraper to get product properties from HTML markup on merchant websites☆19Updated 2 years ago
- Wrapper over node-imap, providing a simpler api for common use cases☆252Updated 4 years ago
- bi-directional request-response for socket.io☆29Updated 7 years ago
- ☆35Updated 2 years ago
- Easy 2-Factor Integration For Node.JS☆220Updated 3 years ago
- Verify email address checking MX records, and SMTP connection.☆123Updated 4 years ago
- Server-side fingerprinting☆137Updated 2 years ago
- NPM package for creating a keyword array from a string and excluding stop words.☆202Updated last year
- Node.js API for Client API☆335Updated 11 months ago
- Node.js API for GeoIP2 webservice client and database reader☆265Updated last week
- Generates the RFC-4122 Name-Based UUID☆67Updated last year
- Crawlee dev repo☆25Updated 3 years ago
- [API v3] Official Mailjet API v3 NodeJS wrapper☆252Updated 3 months ago
- A rate limiting store for express-rate-limit with Redis/Redict/Valkey/etc.☆212Updated 3 weeks ago
- This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the use…☆96Updated this week
- Async Node JS module to check if an email address exists☆80Updated 3 years ago
- 📳 Mercure Hub & Publisher implemented in node.js.☆34Updated last month
- The most efficient JS implementation calculating the Levenshtein distance, i.e. the difference between two strings.☆507Updated 3 years ago
- socket.io v2.x with a wildcard event☆198Updated last month
- Push notifications for GCM, APNS, MPNS, AMZ (automatic detection from device token)☆549Updated 3 weeks ago
- JavaScript tool to unserialize data taken from PHP. It can parse "serialize()" output, or even serialized sessions data.☆142Updated 5 years ago
- ISO-639-1 codes☆220Updated 8 months ago
- Serverless PDF generator☆60Updated 4 years ago