get-set-fetch / scraperLinks
Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.
☆114Updated 2 years ago
Alternatives and similar repositories for scraper
Users that are interested in scraper are comparing it to the libraries listed below
Sorting:
- web scraping extension☆84Updated last month
- Cloud crawler functions for scrapeulous☆45Updated 4 years ago
- Man in the middle using Playwright☆27Updated 2 years ago
- House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.☆121Updated 2 years ago
- 🧱 A uniform template to use as a foundation for Puppeteer bot construction.☆67Updated 4 years ago
- NodeJs package for generating browser-like headers.☆72Updated 2 years ago
- A tiny demo, showing how to build your own scriptable HTTPS-intercepting proxy with Mockttp☆23Updated 2 years ago
- All In One API to easily scrape data from any website, without worrying about captchas and bot detection mecanisms.☆22Updated last year
- Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other …☆30Updated 3 years ago
- Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt☆82Updated last year
- A simple puppeteer wrapper to enable useful plugins with ease☆57Updated this week
- A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppetee…☆96Updated 2 years ago
- An undetectable browser automation framework 🤖☆33Updated 3 years ago
- Utilities and constants shared across Apify projects.☆14Updated this week
- Playwright Docker Images. (Ubuntu, Alpine) x (ARMv8, x64) x (Chromium, Firefox, WebKit, Chrome, Edge)☆60Updated 4 months ago
- Scrape SEO elements or whatever you need with this scraper built in Node.js☆38Updated 3 years ago
- Parses OTP messages for a verification code and service provider.☆24Updated 2 years ago
- A puppeteer-extra plugin to remotely view and interact with puppeteer sessions. Essentially opening a "portal" to the page.☆53Updated 2 years ago
- You can use this act to monitor any page's content and get a notification when content changes.☆20Updated 2 years ago
- ☆12Updated 7 years ago
- Generate a list of keywords from any text.☆31Updated 5 years ago
- This GitHub project is a scraper specifically designed to extract all Facebook ads from the Facebook Ads Library. The scraper is capable …☆25Updated last year
- In-Memory Key-Value Database with Persistent File Storage☆16Updated 2 years ago
- automatic and extensive scraper for forums☆25Updated 2 weeks ago
- DFPM is a browser extension for detecting browser fingerprinting.☆119Updated 2 years ago
- THE LOCAL SEO DOMINATOR - CONTENT MANAGEMENT SYSTEM AND SITEMAP MODULE The Local SEO Dominator is a light-weight content management syst…☆23Updated 5 years ago
- Home of the Ulixee Open Data Platform☆55Updated 3 weeks ago
- Minimal set of tools to conduct stealthy scraping.☆156Updated 2 years ago
- 🕺 Humanizer functions for Puppeteer☆37Updated last year
- Phantombuster's SDK☆14Updated 8 months ago