brendonboshell / supercrawlerLinks
A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
☆382Updated 3 years ago
Alternatives and similar repositories for supercrawler
Users that are interested in supercrawler are comparing it to the libraries listed below
Sorting:
- Node.js email SMTP verification, powered by EmailChecker.com API☆296Updated 3 months ago
- Google Search SERP Scraper☆125Updated last week
- Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)☆498Updated 5 years ago
- Web crawler for Node.JS☆256Updated 7 years ago
- Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues.☆435Updated 3 years ago
- House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.☆128Updated 3 weeks ago
- Email automation driven by headless chrome.☆167Updated 5 years ago
- Blazingly fast, multi tenant, faceted search API☆312Updated 5 years ago
- Javascript scraping module based on puppeteer for many different search engines...☆567Updated 3 years ago
- Nodejs lib to parse Google SERP html pages☆47Updated 2 years ago
- Verify email address checking MX records, and SMTP connection.☆123Updated 4 years ago
- Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSO…☆153Updated 2 years ago
- A look at how LinkedIn spies on its users.☆832Updated 7 years ago
- simple multi-level scraper json input/output for Cheerio☆199Updated 3 years ago
- A Node.js module to search and scrape Google.☆456Updated 7 years ago
- Module that extracts structured information from a rendered html site and outputs JSON. HTML to JSON.☆70Updated 4 years ago
- High-performance FlexSearch Server for Node.js (Cluster)☆190Updated 6 years ago
- Simple, lightweight and expressive web scraping with Node.js☆153Updated 4 years ago
- Crawler for LinkedIn full profiles 2019☆216Updated 5 years ago
- Chromium / Puppeteer site crawler☆48Updated 5 years ago
- LinkedIn Scraper (currently working 2020)☆610Updated 2 years ago
- Google search scraper with captcha solving support☆90Updated 6 years ago
- Cloud crawler functions for scrapeulous☆45Updated 4 years ago
- Example project demonstrating Headless Chrome + Puppeteer running in their own individual containers.☆70Updated 3 years ago
- A light, fast and flexible javascript tracking library☆263Updated 2 years ago
- An AliExpress spider for Node☆45Updated 8 years ago
- Amazon crawler - this configuration will extract items for a keywords that you will specify in the input, and it will automatically extra…☆76Updated 5 years ago
- Instagram automation driven by headless chrome.☆119Updated 3 years ago
- Node.js price monitoring library, leveraging the power of x-ray and nightmare.☆55Updated 6 years ago
- Create dynamic social share images using HTML + CSS via puppeteer 🎁☆217Updated 3 years ago