samclarke / robots-parserLinks
NodeJS robots.txt parser with support for wildcard (*) matching.
☆164Updated last month
Alternatives and similar repositories for robots-parser
Users that are interested in robots-parser are comparing it to the libraries listed below
Sorting:
- A fast, safe, compliant XML parser for Node.js and browsers.☆316Updated last year
- A cacheable dns.lookup(…) that respects TTL☆217Updated last year
- A module for node.js and the browser that takes in text and strips it of stopwords☆261Updated 2 weeks ago
- A library for colorizing JSON strings☆88Updated last year
- Streaming line delimited json parser + serializer☆129Updated 4 years ago
- A high-performance, lightweight LRU cache. Built for developers who need fast caching without compromising on features.☆174Updated last week
- Splits a hostname into subdomains, domain and (effective) top-level domains.☆514Updated 2 months ago
- Obscene words filter for nanoid☆90Updated 3 years ago
- Language detection for Javascript (Node). Based on the CLD2 (Compact Language Detector) library from Google.☆334Updated 10 months ago
- Node.js DNS over HTTPS - Tangerine is the best drop-in replacement for dns.promises.Resolver using DNS over HTTPS ("DoH") via undici wit…☆178Updated this week
- a monorepo of npm packages☆207Updated 2 months ago
- Run promise-returning & async functions concurrently with optional limited concurrency☆344Updated 3 months ago
- Character encoding detection tool for NodeJS☆300Updated last month
- Safely and quickly serialize JavaScript objects☆356Updated last year
- Rate limit made simple, easy, async.☆336Updated last month
- A queue with custom concurrency and time limit.☆196Updated 3 years ago
- ☆94Updated 10 months ago
- Compare strings containing a mix of letters and numbers in the way a human being would in sort order.☆113Updated last year
- Convert Unicode characters to Latin characters using transliteration☆316Updated last week
- Parser for RFC2822 (Header) format email addresses☆55Updated 6 months ago
- Check where your code is running. In the browser or in node.js environment.☆252Updated last year
- Node.js framework agnostic library that enables you to forward an http request to another HTTP server. Supported protocols: HTTP, HTTPS, …☆162Updated last year
- making fetch happen for npm☆204Updated 2 months ago
- JSON.parse() drop-in replacement with prototype poisoning protection☆239Updated last month
- NPM open registry☆72Updated last month
- Node.js implementation of HighwayHash, Google's fast and strong hash function☆221Updated 4 years ago
- 🚸 Cross platform children list of a PID.☆126Updated 3 years ago
- Parse and stringify BCP 47 language tags☆74Updated 2 years ago
- It parses the accept-language header from an HTTP request☆166Updated 3 years ago
- 🇫🇷 NodeJS language detection library using n-gram☆409Updated 5 years ago