Alternatives and similar repositories for robots-parser:

Users that are interested in robots-parser are comparing it to the libraries listed below

• peerigon / parse-domain
  Splits a hostname into subdomains, domain and (effective) top-level domains.
  ☆511Updated 3 months ago
• szmarczak / cacheable-lookup
  A cacheable dns.lookup(…) that respects TTL
  ☆203Updated last year
• avoidwork / tiny-lru
  Tiny LRU cache for Client or Server
  ☆161Updated this week
• forwardemail / nodejs-dns-over-https-tangerine
  Node.js DNS over HTTPS - Tangerine is the best drop-in replacement for dns.promises.Resolver using DNS over HTTPS ("DoH") via undici wit…
  ☆164Updated 4 months ago
• ekalinin / robots.js
  Parser for robots.txt for node.js
  ☆67Updated 4 years ago
• fent / timequeue.js
  A queue with custom concurrency and time limit.
  ☆197Updated 3 years ago
• ndjson / ndjson.js
  Streaming line delimited json parser + serializer
  ☆119Updated 3 years ago
• sindresorhus / get-stream
  Get a stream as a string, Buffer, ArrayBuffer or array
  ☆347Updated 5 months ago
• codsen / codsen
  a monorepo of npm packages
  ☆196Updated last month
• delvedor / hpagent
  A ready to use http and https agent for working with proxies that keeps connections alive!
  ☆185Updated last year
• microlinkhq / async-ratelimiter
  Rate limit made simple, easy, async.
  ☆331Updated last week
• runk / node-chardet
  Character encoding detection tool for NodeJS
  ☆288Updated last month
• sindresorhus / p-all
  Run promise-returning & async functions concurrently with optional limited concurrency
  ☆328Updated 4 months ago
• joeattardi / json-colorizer
  A library for colorizing JSON strings
  ☆85Updated 11 months ago
• fb55 / entities
  encode & decode HTML & XML entities with ease & speed
  ☆338Updated last week
• isaacs / ttlcache
  ☆183Updated last year
• fergiemcdowall / stopword
  A module for node.js and the browser that takes in text and strips it of stopwords
  ☆246Updated 3 months ago
• sindresorhus / get-urls
  Get all urls in a string
  ☆368Updated last year
• aduth / memize
  Unabashedly-barebones memoization library with an aim toward speed
  ☆119Updated last year
• sindresorhus / srcset
  Parse and stringify the HTML <img> srcset attribute
  ☆139Updated last year
• lupomontero / psl
  JavaScript domain name parser based on the Public Suffix List
  ☆407Updated last week
• nodeca / url-unshort
  Short links expander for node.js
  ☆120Updated last year
• lovell / highwayhash
  Node.js implementation of HighwayHash, Google's fast and strong hash function
  ☆219Updated 3 years ago
• sindresorhus / exit-hook
  Run some code when the process exits
  ☆284Updated last year
• rgrove / parse-xml
  A fast, safe, compliant XML parser for Node.js and browsers.
  ☆307Updated 5 months ago
• yosuke-furukawa / server-timing
  This module adds [Server-Timing](https://www.w3.org/TR/server-timing/) to response headers, see [example](https://server-timing.now.sh/) …
  ☆126Updated 3 weeks ago
• morteza-fsh / puppeteer-full-page-screenshot
  ☆90Updated 2 months ago
• sindresorhus / p-memoize
  Memoize promise-returning & async functions
  ☆405Updated last year
• 75lb / wordwrapjs
  Word-wrapping for javascript.
  ☆161Updated 5 months ago
• mykolaharmash / hyntax
  Straightforward HTML parser for JavaScript
  ☆139Updated 9 months ago