lorien / awesome-web-scraping
List of libraries, tools and APIs for web scraping and data processing.
☆6,699Updated 3 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for awesome-web-scraping
- A collection of awesome web crawler,spider in different languages☆6,488Updated 5 months ago
- Web Scraping Framework☆2,393Updated 8 months ago
- 🔍 A helpful checklist/collection of Search Engine Optimization (SEO) tips and techniques.☆2,438Updated 6 months ago
- A curated list of amazingly awesome open-source sysadmin resources.☆25,533Updated 3 months ago
- Visual scraping for Scrapy☆9,303Updated 4 months ago
- Awesome list of GraphQL☆14,598Updated this week
- A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.☆2,643Updated 3 years ago
- Lightweight, scriptable browser as a service with an HTTP API☆4,099Updated 3 months ago
- A curated list of awesome PostgreSQL software, libraries, tools and resources, inspired by awesome-mysql☆10,067Updated 2 months ago
- A pure-python HTML screen-scraping library☆1,863Updated 2 years ago
- A curated list of amazingly awesome open source sysadmin resources inspired by Awesome PHP.☆23,751Updated 7 months ago
- A curated list of awesome packages, articles, and other cool resources from the Scrapy community.☆537Updated last year
- A curated list of awesome services, solutions and resources for serverless / nobackend applications.☆7,451Updated 6 months ago
- A curated list of analytics frameworks, software and other tools.☆3,940Updated 6 months ago
- A service daemon to run Scrapy spiders☆2,970Updated last week
- The most awesome list about bots ⭐️🤖☆3,830Updated 4 months ago
- A list of scrapers from around the web.☆625Updated 3 months ago
- admin ui for scrapy/open source scrapinghub☆2,741Updated last year
- A collaborative list of great resources about RESTful API architecture, development, test, and performance☆3,647Updated 2 months ago
- Distributed crawler powered by Headless Chrome☆5,528Updated last year
- Compiled list of links from "Ask HN: Where can I post my startup to get beta users?"☆5,976Updated last month
- A list of (almost) all headless web browsers in existence☆6,242Updated 5 months ago
- A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fier…☆12,503Updated 8 months ago
- Web crawling framework based on asyncio.☆2,035Updated 5 years ago
- A scalable frontier for web crawlers☆1,302Updated last year
- A curated list of awesome command-line frameworks, toolkits, guides and gizmos. Inspired by awesome-php.☆33,171Updated 3 months ago
- Use your terminal shell to do awesome things.☆3,962Updated 3 years ago
- Pythonic HTML Parsing for Humans™☆13,740Updated 7 months ago
- Awesome tooling and resources in the Chrome DevTools & DevTools Protocol ecosystem☆6,137Updated 2 months ago
- A collection of links for free stock photography, video and Illustration websites☆13,046Updated 4 months ago