get-set-fetch / scraper
Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.
β112Updated 2 years ago
Alternatives and similar repositories for scraper:
Users that are interested in scraper are comparing it to the libraries listed below
- π§± A uniform template to use as a foundation for Puppeteer bot construction.β66Updated 3 years ago
- Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other β¦β30Updated 3 years ago
- web scraping extensionβ81Updated last month
- House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.β121Updated 2 years ago
- Base Docker images for Apify actors.β76Updated this week
- A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteeβ¦β94Updated 2 years ago
- A single tab web browser built with puppeteer. Also, no client-side JS. Viewport is streamed with MJPEG. For realz.β56Updated last year
- A simple puppeteer wrapper to enable useful plugins with easeβ56Updated this week
- NodeJs package for generating browser-like headers.β69Updated 2 years ago
- Cloud crawler functions for scrapeulousβ45Updated 4 years ago
- Aliexpress.com scraper which developed for Apifyβ1Updated 2 months ago
- A suite of tools for protecting the web's open knowledge.β127Updated 7 months ago
- Extracts email address from an arbitrary text input.β62Updated 2 months ago
- Home of the Ulixee Open Data Platformβ50Updated 4 months ago
- Utilities and constants shared across Apify projects.β14Updated this week
- Phantombuster's SDKβ14Updated 6 months ago
- An undetectable browser automation framework π€β32Updated 3 years ago
- Solve captchas for Puppeteer / Seleniumβ20Updated last year
- Chromium / Puppeteer site crawlerβ48Updated 5 years ago
- πΊ Humanizer functions for Puppeteerβ37Updated last year
- A test suite of common scraper detection techniques. See how detectable your scraper stack is.β137Updated 2 years ago
- Web data extraction tool implemented as chrome extension with much more featuresβ47Updated 6 years ago
- Parses OTP messages for a verification code and service provider.β24Updated 2 years ago
- Generates realistic browser fingerprintsβ76Updated 2 years ago
- Automated functional testing via the Chrome DevTools Protocol. Easy to use and open source. Generates unique CSS and Xpath selectors. Outβ¦β54Updated 4 years ago
- Build Your DXP is an open-source catalog to explore the best-of-breed services that power today's Digital Experience Platforms, enabling β¦β26Updated 2 years ago
- DFPM is a browser extension for detecting browser fingerprinting.β116Updated 2 years ago
- Home of fingerprint injector.β68Updated 2 years ago
- All In One API to easily scrape data from any website, without worrying about captchas and bot detection mecanisms.β22Updated last year
- Add-ons for Playwright: adblocker, stealth modeβ46Updated 4 years ago