Kikobeats / top-crawler-agentsLinks
A list of common crawler user agents useful for retrieving metadata from links.
☆26Updated 2 weeks ago
Alternatives and similar repositories for top-crawler-agents
Users that are interested in top-crawler-agents are comparing it to the libraries listed below
Sorting:
- Find a file or directory by walking up parent directories — Zero dependencies☆80Updated 4 months ago
 - Use "cw" in the CLI. No instructions necessary, just hit <enter>. Can also be used as a library. Commit Whisperer is an AI genius for gen…☆19Updated 11 months ago
 - Drop-dead simple keypress event emitter for Node.js. Create powerful CLI applications and experiences with ease.☆17Updated 4 months ago
 - Create a multicast stream that lets multiple consumers independently read the same data☆181Updated 3 months ago
 - WebExtension packages for every need☆125Updated 7 months ago
 - 🛡️📧 Protect e-mails against spam and scraping bots☆32Updated 9 months ago
 - A curated list of well-known URIs, resources, guides and tools (RFC 5785)☆79Updated last year
 - Observe changes to the DOM using an async iterable — A nicer API for MutationObserver☆222Updated last month
 - Utilities for working with Windows Subsystem for Linux (WSL)☆60Updated 2 months ago
 - Get the operating system CPU architecture☆40Updated last year
 - Encode & decode strings, bytes, and integers to Base62☆91Updated last month
 - Get the HTML from any website, using prerendering when necessary.☆92Updated this week
 - Create "perfect" snapshots of web pages☆33Updated 2 months ago
 - A minimalistic wrapper around Node.js `child_process.spawn` API.☆37Updated 2 weeks ago
 - Utility functions for WebExtensions☆23Updated 9 months ago
 - ☁️ Command Line Interface for Cloudflare R2 Storage☆25Updated last week
 - A SQLite based job queue implementation in typescript☆49Updated last month
 - Check which JavaScript environment your code is running in at runtime: browser, Node.js, Bun, etc☆164Updated last year
 - Make a regular expression time out if it takes too long to execute☆218Updated last year
 - Get and set images on the macOS clipboard☆44Updated 2 months ago
 - Get a random temporary path☆32Updated last year
 - Detects where the current browser extension code is being run. Chrome and Firefox.☆50Updated 9 months ago
 - 💫 A lightweight browser extension to jump to various external bookmarks from the address bar.☆20Updated 5 months ago
 - Dynamically generated currency conversion rate file☆35Updated this week
 - Simplest way to read & normalize RSS/ATOM/JSON feed data☆182Updated last month
 - Fastest, plain JavaScript-based, HTML escaping library for JavaScript, works in both Node.js and browser. Even faster than a Rust-based o…☆37Updated last month
 - Extracts the repo URL from a package.json object☆28Updated 3 months ago
 - Collection of manifest files for 100k Chrome extensions☆84Updated 9 months ago
 - A more usable typed storage API for Web Extensions☆72Updated 7 months ago
 - Capture screenshots programmatically. Cross-platform, with support for MacOS (Darwin), Windows, and Linux.☆43Updated 3 months ago