lorien / awesome-web-scrapingLinks
List of libraries, tools and APIs for web scraping and data processing.
☆7,120Updated 7 months ago
Alternatives and similar repositories for awesome-web-scraping
Users that are interested in awesome-web-scraping are comparing it to the libraries listed below
Sorting:
- A collection of awesome web crawler,spider in different languages☆6,904Updated last year
- Web Scraping Framework☆2,407Updated last year
- Scrapoxy is a super proxies manager that orchestrates all your proxies into one place, rather than spreading management across multiple s…☆2,335Updated this week
- A curated list of awesome packages, articles, and other cool resources from the Scrapy community.☆551Updated 2 years ago
- Lightweight, scriptable browser as a service with an HTTP API☆4,166Updated last year
- A collaborative list of great resources about RESTful API architecture, development, test, and performance☆3,774Updated 2 weeks ago
- The web scraping open project repository aims to share knowledge and experiences about web scraping with Python☆1,665Updated last year
- A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.☆2,722Updated 4 years ago
- A service daemon to run Scrapy spiders☆3,056Updated this week
- A pure-python HTML screen-scraping library☆1,882Updated 3 years ago
- A list of (almost) all headless web browsers in existence☆6,414Updated 4 months ago
- The definitive list of lists (of lists) curated on GitHub and elsewhere☆10,517Updated 2 months ago
- Rotating TOR proxy with Docker☆1,183Updated last year
- Scrapy+Splash for JavaScript integration☆3,218Updated 6 months ago
- A list of scrapers from around the web.☆681Updated 6 months ago
- A a curated list of curated lists of awesome lists.☆2,040Updated last year
- The most awesome list about bots ⭐️🤖☆3,976Updated last year
- A scalable frontier for web crawlers☆1,314Updated 2 months ago
- Tools for building bots☆1,458Updated last year
- newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:☆14,708Updated last month
- A curated list of awesome big data frameworks, ressources and other awesomeness.☆13,794Updated 6 months ago
- 🔍 A helpful checklist/collection of Search Engine Optimization (SEO) tips and techniques.☆2,575Updated 5 months ago
- Awesome tooling and resources in the Chrome DevTools & DevTools Protocol ecosystem☆6,503Updated 7 months ago
- A curated list of analytics frameworks, software and other tools.☆4,116Updated last year
- Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.☆8,832Updated last year
- Web data extraction tool implemented as chrome extension☆1,341Updated 6 years ago
- Find your next book to read!☆12,653Updated 9 months ago
- Design and development guides☆2,299Updated last month
- 🎭 Playwright integration for Scrapy☆1,243Updated last week
- A list of history's greatest software engineers and tech pioneers☆2,521Updated 4 years ago