teticio / lambda-scraperLinks
Use AWS Lambda functions as a proxy pool to scrape web pages.
☆133Updated last year
Alternatives and similar repositories for lambda-scraper
Users that are interested in lambda-scraper are comparing it to the libraries listed below
Sorting:
- The Web Scraping Club Free Repository☆145Updated last month
- A fork of Dragnet that also extract author, headline, date, keywords from context, as well as built in metadata extraction all in one pac…☆284Updated last month
- Minimal set of tools to conduct stealthy scraping.☆156Updated 2 years ago
- Professional scrapers that provide full control to the users. Crawlee One builds on top of Crawlee and Apify and extends them with featur…☆31Updated last year
- Asynchronous alternative to the requests-ip-rotator library☆43Updated 5 months ago
- Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters☆140Updated 5 months ago
- Super Fast, Super Anti-Detect, and Super Intuitive Web Driver☆70Updated last week
- estela, an elastic web scraping cluster 🕸☆184Updated 3 weeks ago
- TypeScript library for Google search scraping using http requests with proxy support, pagination, and regional customization. Built for w…☆39Updated 4 months ago
- The Architecture of a Web Crawler: Building a Google-Inspired Distributed Web Crawler☆117Updated 6 months ago
- A python package for finding e-mails, checking deliverability and more.☆66Updated last year
- Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues.☆430Updated 2 years ago
- Unflare helps you to bypass Cloudflare protection☆116Updated 2 months ago
- Index Common Crawl archives in tabular format☆122Updated last month
- Shopify Scraper package to extract all products from a Shopify site and return them in a Pandas dataframe.☆34Updated last year
- AI based web-wrapper for web-content-extraction☆100Updated 2 years ago
- 🕷️ Scrapyd is an application for deploying and running Scrapy spiders.☆85Updated last month
- Create on demand free HTTPS/SOCKS5 proxy servers using AWS Free Tier EC2 instances automatically with Terraform☆92Updated 2 years ago
- playwright stealth☆707Updated 10 months ago
- Library that helps use puppeteer in scrapy.☆52Updated 3 weeks ago
- Spider ported to Node.js☆46Updated 4 months ago
- 🎭 Intelligent browser header & fingerprint generator☆601Updated 3 months ago
- ☆23Updated 6 months ago
- Zyte Automatic Extraction integration for Scrapy☆56Updated 3 years ago
- A simple LinkedIn profile scraper implemented as a chrome extension☆82Updated 2 weeks ago
- Cloud crawler functions for scrapeulous☆45Updated 4 years ago
- Curated list of everything related to captchas, including libraries, solvers and scoring☆30Updated 11 months ago
- Spider ported to Python☆86Updated 4 months ago
- AI article writer to automatically generate articles with 1,500-7,000+ words to boost your website's SEO and make it more alive☆25Updated last year
- Scrapy download handler that can impersonate browser' TLS signatures or JA3 fingerprints.☆172Updated last month