bda-research / node-webcrawler
Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously
☆41Updated 8 years ago
Alternatives and similar repositories for node-webcrawler:
Users that are interested in node-webcrawler are comparing it to the libraries listed below
- Node.js Client Library for seaweedfs (weed-fs)☆50Updated 2 years ago
- general server render base on headless chrome☆95Updated 7 years ago
- Watches for changes in MongoDB replication log.☆95Updated 10 years ago
- A gulp node-inspector wrapper☆21Updated 8 years ago
- Redis-based task queue library inspired by Celery and Kue.☆56Updated 10 years ago
- Streaming Imagemagick API☆92Updated 4 years ago
- remote monitoring and debugging for socket.io☆452Updated 10 years ago
- A cross-platform thread pool add-on for Node.js and io.js.☆107Updated 8 years ago
- Image optimizer, PNG, JPEG and GIFimage compress on OS X, Linux, FreeBSD and Windows☆67Updated 5 years ago
- HTML5/Node/WebSocket MongoDB panel☆441Updated 9 years ago
- Cache your Mongoose MongoDB query results with Redis. 300% faster queries FTW!☆237Updated 10 years ago
- Middleware to attach your express app's request & response to a domain. No more process.uncaughtException nonsense.☆120Updated 9 years ago
- ❤️ Middleware for easy rendering of async Query results.☆316Updated 9 years ago
- Interactive terminal list for nodejs☆125Updated 11 years ago
- This is the pure Node API for reading MaxMind DB files. MaxMind DB is a binary file format that stores data indexed by IP address subnets…☆87Updated 6 years ago
- etagify is connect middleware to add ETag headers to cachable but non-static content.☆49Updated 13 years ago
- A node.js (>= 0.8.x) compatible multi-process management module☆278Updated 11 years ago
- Smart and easy request rate limiter for your APIs built on `koa`, using `koa-ip-filter`. Support custom stores, custom ID, custom error …☆38Updated 2 years ago
- Redis adapter for SocketCluster☆45Updated 5 years ago
- A node module for transaction-like db writes☆31Updated 7 years ago
- Adds a handy `trace` flag to the console object to prepend the file and line number.☆297Updated 12 years ago
- Run node with ES6 Generators, today!☆247Updated 9 years ago
- Middleware allowing the client to filter the response to only what they need, reducing the amount of traffic over the wire.☆94Updated 7 years ago
- Friendly web crawler for x-ray☆44Updated 2 years ago
- AMQP RPC driver for node. Tested on RabbitMQ.☆49Updated 8 years ago
- phash for images☆55Updated 5 years ago
- Communicate between objects in servers, mobile apps, and other servers.☆106Updated 10 months ago
- Image analysis and comparison☆66Updated 8 years ago
- Promises/A+ conformant implementation☆85Updated 8 years ago
- Action based authorization middleware.☆58Updated 4 years ago