get-set-fetch / scraper
Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.
☆111Updated 2 years ago
Alternatives and similar repositories for scraper:
Users that are interested in scraper are comparing it to the libraries listed below
- A single tab web browser built with puppeteer. Also, no client-side JS. Viewport is streamed with MJPEG. For realz.☆56Updated last year
- Cloud crawler functions for scrapeulous☆45Updated 4 years ago
- web scraping extension☆81Updated 3 weeks ago
- All In One API to easily scrape data from any website, without worrying about captchas and bot detection mecanisms.☆21Updated last year
- Parses OTP messages for a verification code and service provider.☆24Updated 2 years ago
- House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.☆121Updated last year
- Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt☆81Updated last year
- Object storage microservice. Like minio but minnier.☆9Updated 5 years ago
- ☆11Updated 7 years ago
- Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other …☆30Updated 3 years ago
- You can use this act to monitor any page's content and get a notification when content changes.☆20Updated 2 years ago
- A puppeteer-extra plugin to remotely view and interact with puppeteer sessions. Essentially opening a "portal" to the page.☆51Updated last year
- Web scraper using Cloudflare Workers☆23Updated 3 years ago
- The ultimate tool for obtaining free proxies from multiple sources and storing them in a MongoDB database.☆21Updated last year
- A tiny demo, showing how to build your own scriptable HTTPS-intercepting proxy with Mockttp☆23Updated 2 years ago
- Aliexpress.com scraper which developed for Apify☆1Updated 2 months ago
- https://meta.mehari.workers.dev☆22Updated 2 years ago
- A self-hosted dashboard and API to share service ports with the team.☆32Updated 2 years ago
- A dead simple web-clipper | ✂Capture ⇒ ⊞ Select ⇒ ✔Done☆32Updated 7 years ago
- Utilities and constants shared across Apify projects.☆14Updated 3 weeks ago
- DronaHQ offers a low-code platform to build internal tools. Drag-and-drop UI components and connect them to your databases and APIs to bu…☆58Updated last month
- Etsy API wrapper written in typescript☆39Updated last year
- command line Google search and save to JSON☆106Updated last year
- Amazon affiliate link storefront powered by a Reddit scraper☆14Updated 7 years ago
- This GitHub project is a scraper specifically designed to extract all Facebook ads from the Facebook Ads Library. The scraper is capable …☆21Updated last year
- Web data extraction tool implemented as chrome extension with much more features☆47Updated 6 years ago
- Base Docker images for Apify actors.☆77Updated this week
- Phantombuster's SDK☆14Updated 5 months ago
- Man in the middle using Playwright☆26Updated 2 years ago
- This project experiments with the Google NLP Algorithm to evaluate e-commerce product descriptions from an SEO perspective.☆17Updated 4 years ago