Alternatives and similar repositories for robots-parser:

Users that are interested in robots-parser are comparing it to the libraries listed below

• sindresorhus / p-all
  Run promise-returning & async functions concurrently with optional limited concurrency
  ☆326Updated 2 months ago
• fergiemcdowall / stopword
  A module for node.js and the browser that takes in text and strips it of stopwords
  ☆243Updated 2 months ago
• szmarczak / cacheable-lookup
  A cacheable dns.lookup(…) that respects TTL
  ☆199Updated last year
• peerigon / parse-domain
  Splits a hostname into subdomains, domain and (effective) top-level domains.
  ☆511Updated last month
• ekalinin / robots.js
  Parser for robots.txt for node.js
  ☆67Updated 3 years ago
• forwardemail / nodejs-dns-over-https-tangerine
  Node.js DNS over HTTPS - Tangerine is the best drop-in replacement for dns.promises.Resolver using DNS over HTTPS ("DoH") via undici wit…
  ☆161Updated 2 months ago
• perezd / node-murmurhash
  A Node.js module for the optimized JavaScript implementation of the MurmurHash algorithms.
  ☆96Updated last year
• sindresorhus / get-urls
  Get all urls in a string
  ☆367Updated last year
• philcockfield / file-system-cache
  A super-fast, promise based cache that reads and writes to the file-system.
  ☆63Updated 7 months ago
• sindresorhus / exit-hook
  Run some code when the process exits
  ☆282Updated last year
• FGRibreau / node-language-detect
  🇫🇷 NodeJS language detection library using n-gram
  ☆400Updated 4 years ago
• hapijs / bourne
  JSON.parse() drop-in replacement with prototype poisoning protection
  ☆173Updated 4 months ago
• npm / make-fetch-happen
  making fetch happen for npm
  ☆192Updated 3 months ago
• fastify / secure-json-parse
  JSON.parse() drop-in replacement with prototype poisoning protection
  ☆229Updated this week
• avoidwork / tiny-lru
  Tiny LRU cache for Client or Server
  ☆159Updated this week
• nodeca / url-unshort
  Short links expander for node.js
  ☆120Updated last year
• sindresorhus / get-stream
  Get a stream as a string, Buffer, ArrayBuffer or array
  ☆345Updated 4 months ago
• devongovett / exif-reader
  A small EXIF image metadata reader
  ☆159Updated 2 months ago
• tinyhttp / milliparsec
  🌌 Tiniest body parser in the universe. Built for modern Node.js
  ☆195Updated 2 weeks ago
• taskforcesh / nandu
  NPM open registry
  ☆68Updated 2 years ago
• sindresorhus / uint8array-extras
  Useful utilities for working with Uint8Array (and Buffer)
  ☆251Updated last month
• sindresorhus / srcset
  Parse and stringify the HTML <img> srcset attribute
  ☆138Updated last year
• mcollina / fast-json-parse
  The fastest way to parse JSON safely
  ☆88Updated 3 years ago
• flash-oss / lil-http-terminator
  Zero dependencies, gracefully terminates HTTP(S) server.
  ☆18Updated last year
• delvedor / hpagent
  A ready to use http and https agent for working with proxies that keeps connections alive!
  ☆185Updated last year
• gjtorikian / isBinaryFile
  Detects if a file is binary in Node.js. Similar to Perl's -B
  ☆164Updated 4 months ago
• sindresorhus / make-asynchronous
  Make a synchronous function asynchronous by running it in a worker
  ☆249Updated 5 months ago
• rgrove / parse-xml
  A fast, safe, compliant XML parser for Node.js and browsers.
  ☆305Updated 4 months ago
• fastify / fast-uri
  Dependency-free RFC 3986 URI toolbox
  ☆91Updated this week
• runk / node-geolite2
  Maxmind's GeoLite2 Free Databases Helper
  ☆63Updated 2 months ago