bda-research / node-webcrawler
Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously
☆41Updated 8 years ago
Alternatives and similar repositories for node-webcrawler
Users that are interested in node-webcrawler are comparing it to the libraries listed below
Sorting:
- Native javascript implementation of the standard Sphinx API☆76Updated 6 years ago
- Image optimizer, PNG, JPEG and GIFimage compress on OS X, Linux, FreeBSD and Windows☆67Updated 5 years ago
- Node.js Client Library for seaweedfs (weed-fs)☆50Updated 2 years ago
- Watches for changes in MongoDB replication log.☆95Updated 10 years ago
- phash for images☆55Updated 5 years ago
- Friendly web crawler for x-ray☆44Updated 2 years ago
- Smart and easy request rate limiter for your APIs built on `koa`, using `koa-ip-filter`. Support custom stores, custom ID, custom error …☆38Updated 2 years ago
- REST interface for Redis-Simple-Message-Queue☆43Updated 8 years ago
- A cross-platform thread pool add-on for Node.js and io.js.☆107Updated 8 years ago
- Node.js library for the Gearman distributed job system with support for multiple servers☆100Updated 8 years ago
- Adds a handy `trace` flag to the console object to prepend the file and line number.☆297Updated 12 years ago
- Middleware to attach your express app's request & response to a domain. No more process.uncaughtException nonsense.☆120Updated 9 years ago
- A full-featured httpp/http proxy to work with Nodejs-Httpp☆43Updated last year
- Help developers build their own nw.js apps for different platforms such as win, mac and linux with gulp.☆25Updated 8 years ago
- A 2nd generation spider to crawl any article site, automatic read title and article.☆43Updated 9 years ago
- A node.js module to implement a recommender engine with popular machine-learning algorithms.☆61Updated 9 years ago
- Node.js client for PredictionIO☆17Updated 9 years ago
- A node.js (>= 0.8.x) compatible multi-process management module☆278Updated 11 years ago
- High-level validation rules for JavaScript, based on validator.js☆151Updated 2 years ago
- 🍎 A node module to easily build and publish NW.js apps on the Mac App Store.☆28Updated 9 years ago
- Node.js app for slicing and dicing ~100,000 models in <5ms with easy sorting, paginating, and filtering.☆202Updated 7 years ago
- scws(Simple Chinese Word Split) node.js binding - scws中文分词node.js模块☆13Updated 6 years ago
- High-availability redis in Node.js.☆154Updated 6 years ago
- x-ray's selector parser.☆16Updated 9 years ago
- node module: generate and validate mongodb objectId strings without dependencies☆22Updated 4 years ago
- Redis-based task queue library inspired by Celery and Kue.☆56Updated 11 years ago
- remote monitoring and debugging for socket.io☆452Updated 10 years ago
- general server render base on headless chrome☆95Updated 7 years ago
- Middleware allowing the client to filter the response to only what they need, reducing the amount of traffic over the wire.☆94Updated 7 years ago
- vhost for koajs☆20Updated 10 years ago