bda-research / node-webcrawler
Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously
☆41Updated 8 years ago
Alternatives and similar repositories for node-webcrawler:
Users that are interested in node-webcrawler are comparing it to the libraries listed below
- Image optimizer, PNG, JPEG and GIFimage compress on OS X, Linux, FreeBSD and Windows☆67Updated 4 years ago
- Node.js Client Library for seaweedfs (weed-fs)☆50Updated 2 years ago
- general server render base on headless chrome☆95Updated 7 years ago
- Cache your Mongoose MongoDB query results with Redis. 300% faster queries FTW!☆237Updated 10 years ago
- nChart for node.js inspired by Chart.js.☆48Updated 8 years ago
- Middleware allowing the client to filter the response to only what they need, reducing the amount of traffic over the wire.☆94Updated 6 years ago
- Promises/A+ conformant implementation☆85Updated 7 years ago
- High-level validation rules for JavaScript, based on validator.js☆151Updated 2 years ago
- A node.js module to implement a recommender engine with popular machine-learning algorithms.☆61Updated 9 years ago
- A Koa.js middleware for performing internal batch requests☆36Updated 5 years ago
- A gulp node-inspector wrapper☆21Updated 7 years ago
- Watches for changes in MongoDB replication log.☆95Updated 10 years ago
- High-availability redis in Node.js.☆154Updated 6 years ago
- Node.js app for slicing and dicing ~100,000 models in <5ms with easy sorting, paginating, and filtering.☆202Updated 7 years ago
- Node.js client for PredictionIO☆17Updated 9 years ago
- x-ray's selector parser.☆16Updated 9 years ago
- A full-featured httpp/http proxy to work with Nodejs-Httpp☆43Updated last year
- promisify mongoose via any promise library☆48Updated 9 years ago
- Integrates winston logging system with your Sails application☆14Updated 7 years ago
- NodeJS library for rate limiting using sliding windows stored in Redis☆78Updated 9 years ago
- Friendly web crawler for x-ray☆44Updated 2 years ago
- Make koa apps check internal status and health. Inspired by express-ping☆36Updated 8 years ago
- Smart and easy request rate limiter for your APIs built on `koa`, using `koa-ip-filter`. Support custom stores, custom ID, custom error …☆38Updated 2 years ago
- Redis standalone caching library for Node.JS and also cache engine for cacheman☆36Updated 2 years ago
- Strongloop Loopback connector for Elasticsearch☆33Updated 9 years ago
- AMQP RPC driver for node. Tested on RabbitMQ.☆49Updated 8 years ago
- Adds `markdown` property to Nodemailer e-mail data☆37Updated 5 years ago
- This is the pure Node API for reading MaxMind DB files. MaxMind DB is a binary file format that stores data indexed by IP address subnets…☆87Updated 6 years ago
- Interactive terminal list for nodejs☆125Updated 11 years ago
- node.js wrapper for the Diffbot API (article and frontpage)☆35Updated 9 years ago