bda-research / node-webcrawler
Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously
☆41Updated 8 years ago
Alternatives and similar repositories for node-webcrawler:
Users that are interested in node-webcrawler are comparing it to the libraries listed below
- Node.js Client Library for seaweedfs (weed-fs)☆50Updated 2 years ago
- Watches for changes in MongoDB replication log.☆95Updated 9 years ago
- Strongloop Loopback connector for Elasticsearch☆33Updated 9 years ago
- Node.js library for the Gearman distributed job system with support for multiple servers☆101Updated 7 years ago
- A Koa.js middleware for performing internal batch requests☆36Updated 5 years ago
- Native javascript implementation of the standard Sphinx API☆76Updated 5 years ago
- Node.js app for slicing and dicing ~100,000 models in <5ms with easy sorting, paginating, and filtering.☆202Updated 7 years ago
- Promises/A+ conformant implementation☆86Updated 7 years ago
- A better array for the browser and node.js. Supports events & many functional goodies.☆710Updated last year
- etagify is connect middleware to add ETag headers to cachable but non-static content.☆49Updated 12 years ago
- REST interface for Redis-Simple-Message-Queue☆43Updated 8 years ago
- A node module for transaction-like db writes☆31Updated 7 years ago
- A gulp node-inspector wrapper☆21Updated 7 years ago
- Friendly web crawler for x-ray☆44Updated 2 years ago
- A pm2 application container for docker.☆75Updated 7 years ago
- Smart and easy request rate limiter for your APIs built on `koa`, using `koa-ip-filter`. Support custom stores, custom ID, custom error …☆38Updated 2 years ago
- A tiny middleware of RESTful API for koa.☆43Updated 7 years ago
- Plugin to get and set alias for fields in Mongoose schemas to reduce db memory footprint☆56Updated 7 years ago
- Simple captcha for Node.JS and Express.☆136Updated 6 years ago
- x-ray's selector parser.☆16Updated 8 years ago
- Image analysis and comparison☆66Updated 8 years ago
- Redis-based task queue library inspired by Celery and Kue.☆56Updated 10 years ago
- Redis adapter for SocketCluster☆45Updated 5 years ago
- ❤️ Middleware for easy rendering of async Query results.☆315Updated 9 years ago
- remote monitoring and debugging for socket.io☆451Updated 10 years ago
- general server render base on headless chrome☆95Updated 7 years ago
- A Knex.js backend for node_acl☆46Updated 2 years ago
- 🍎 A node module to easily build and publish NW.js apps on the Mac App Store.☆28Updated 8 years ago
- A cross-platform thread pool add-on for Node.js and io.js.☆108Updated 8 years ago
- High-level validation rules for JavaScript, based on validator.js☆151Updated 2 years ago