teticio / lambda-scraper
Use AWS Lambda functions as a proxy pool to scrape web pages.
☆112Updated 10 months ago
Related projects ⓘ
Alternatives and complementary repositories for lambda-scraper
- Minimal set of tools to conduct stealthy scraping.☆150Updated last year
- estela, an elastic web scraping cluster 🕸☆173Updated last week
- The Web Scraping Club Free Repository☆128Updated 2 weeks ago
- A fork of Dragnet that also extract author, headline, date, keywords from context, as well as built in metadata extraction all in one pac…☆239Updated 10 months ago
- Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues.☆415Updated last year
- Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters☆126Updated 3 weeks ago
- Library that helps use puppeteer in scrapy.☆52Updated this week
- Web scraping API for building AI applications.☆40Updated 9 months ago
- Open source SaaS metrics (built by Paper)☆156Updated 3 years ago
- Shopify Scraper package to extract all products from a Shopify site and return them in a Pandas dataframe.☆29Updated last year
- This script can help you to submit URLs in bulk to the Google Indexing API.☆34Updated last year
- A simple LinkedIn profile scraper implemented as a chrome extension☆74Updated last year
- Cloud crawler functions for scrapeulous☆44Updated 3 years ago
- This repository provides usage examples for the Python module Newspaper3k.☆142Updated 10 months ago
- Get structured JSON data from any page.☆175Updated last year
- A python package for finding e-mails, checking deliverability and more.☆48Updated 6 months ago
- Module that extracts structured information from a rendered html site and outputs JSON. HTML to JSON.☆69Updated 3 years ago
- Serverless selenium which dynamically execute any given code.☆56Updated 2 years ago
- Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Support…☆107Updated last year
- G2 Scraper helps you collect G2 product data, including names, product descriptions, reviews, ratings, comparisons, alternatives, and mor…☆37Updated 3 months ago
- A basic python 3 based web scraper for extracting reviews from Amazon. Built using Selectorlib and requests☆61Updated 8 months ago
- A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppetee…☆90Updated 2 years ago
- Script that takes any long form video or podcast and outputs clips for social media☆103Updated 10 months ago
- Fully working applications that demonstrate how to use Haystack to implement common NLP use cases☆107Updated last week
- create your rotating proxy server with docker. self hosted rotating proxy service.☆172Updated last year
- Zyte Automatic Extraction integration for Scrapy☆55Updated 2 years ago
- Python SEO keywords suggestion tool. Google Autocomplete, People Also Ask and Related Searches.☆109Updated last year
- ☆72Updated this week