ReedD / crawlerLinks
Chromium / Puppeteer site crawler
☆48Updated 5 years ago
Alternatives and similar repositories for crawler
Users that are interested in crawler are comparing it to the libraries listed below
Sorting:
- Email automation driven by headless chrome.☆167Updated 5 years ago
- House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.☆129Updated 3 weeks ago
- ExpressJs middleware for rendering PWA to bots using Puppeteer.☆121Updated last month
- Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Support…☆112Updated 2 years ago
- Easily generate animated GIFs from websites☆106Updated 2 years ago
- Library and CLI for automating captcha verification across multiple providers.☆121Updated 5 years ago
- Robust text renderer using headless chrome.☆66Updated 2 years ago
- Instagram get images 🌄 (hashtags, account, locations) with puppeteer☆79Updated last month
- Extracts email address from an arbitrary text input.☆64Updated 11 months ago
- A Better Scraper, with Puppeteer☆43Updated last week
- ⛏ A versatile Web scraper for Node.js☆46Updated last month
- A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and con…☆381Updated 3 years ago
- 🎠 Run headless Chrome (aka Puppeteer) as a service.☆50Updated 7 years ago
- Node.JS library and cli for scraping websites using Puppeteer (or not) and YAML definitions☆49Updated 3 years ago
- Convenience functions for the Puppeteer☆25Updated 2 years ago
- Wish your facebook friends from command line, never miss the wish! http://npm.im/facebook-birthday-cli☆58Updated 6 years ago
- Query multiple APIs and DBs and join them in a single query☆136Updated 2 years ago
- Google Search SERP Scraper☆122Updated 2 months ago
- Nodejs lib to parse Google SERP html pages☆47Updated 2 years ago
- Generate HAR file with puppeteer☆166Updated last year
- Example project demonstrating Headless Chrome + Puppeteer running in their own individual containers.☆70Updated 3 years ago
- Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other …☆31Updated 3 years ago
- HTML5 Canvas implementation for NodeJS backed by Puppeteer☆65Updated 2 years ago
- Language agnostic named entity recognizer☆41Updated 2 years ago
- ☆79Updated 11 years ago
- ☆25Updated 4 years ago
- Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues.☆435Updated 3 years ago
- A `htmlparser2` handler for parsing rich metadata from HTML. Includes HTML metadata, JSON-LD, RDFa, microdata, OEmbed, Twitter cards and …☆56Updated 2 years ago
- ⚡️ Next-generation data transformation framework for TypeScript that puts developer experience first☆53Updated 3 years ago
- Simple, lightweight and expressive web scraping with Node.js☆153Updated 4 years ago