chrisakroyd / robots-txt-parserLinks
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
☆13Updated 2 years ago
Alternatives and similar repositories for robots-txt-parser
Users that are interested in robots-txt-parser are comparing it to the libraries listed below
Sorting:
- NodeJS robots.txt parser with support for wildcard (*) matching.☆156Updated 7 months ago
- Small javascript library/function to prettify XML☆13Updated 8 years ago
- bi-directional request-response for socket.io☆30Updated 7 years ago
- Server-side fingerprinting☆134Updated last year
- 📳 Mercure Hub & Publisher implemented in node.js.☆34Updated 2 months ago
- Crawlee dev repo☆22Updated 2 years ago
- Easy 2-Factor Integration For Node.JS☆220Updated 2 years ago
- Proxy your http requests to another server, with hooks☆365Updated this week
- Variable extension for tiptap by Talqui☆11Updated 2 months ago
- Searchable Encryption for Node.js projects☆67Updated 4 months ago
- A module for node.js and the browser that takes in text and strips it of stopwords☆251Updated 2 weeks ago
- On-the-fly image resizing using Node.js and libvips. Heroku Ready!☆73Updated 7 years ago
- Utility library for building tastier errors and stack traces in JavaScript.☆22Updated 5 years ago
- A cryptographically secure generator for random numbers in a range.☆82Updated 3 years ago
- Less memory usage version of geoip-lite by supporting only country lookup.☆92Updated last week
- Chain functions as transform streams.☆92Updated last month
- Models a robots.txt file☆18Updated 5 years ago
- Override the IP address returned for one or more domains.☆25Updated last year
- Parser for robots.txt for node.js☆67Updated 4 years ago
- Streaming line delimited json parser + serializer☆119Updated 3 years ago
- Node.js Cache Manager store for filesystem that saves the data in a file named with the hash of the key☆38Updated 9 months ago
- A php port of Neil Frasers diff_match_patch☆60Updated 10 years ago
- Zero dependencies, gracefully terminates HTTP(S) server.☆21Updated 2 years ago
- A cacheable dns.lookup(…) that respects TTL☆208Updated last year
- consistent hashing in javascript☆33Updated last year
- Nodejs TLD extract☆46Updated 11 months ago
- Mongo-like queries for Javascript object arrays☆92Updated 10 months ago
- A streaming parser for sitemap files. Is able to deal with deeply nested sitemaps with 100+ million urls in them.☆39Updated 3 years ago
- Fastify Plugin to serve responses for health checks☆40Updated 7 months ago
- A list of common eMail providers.☆46Updated 11 months ago