🕶 Awesome list of Scrapy tools and libraries
☆60Jul 6, 2020Updated 6 years ago
Alternatives and similar repositories for awesome-scrapy
Users that are interested in awesome-scrapy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A library to make it easier to load input URLs to start scrapy processes☆14Feb 21, 2021Updated 5 years ago
- A curated list of awesome packages, articles, and other cool resources from the Scrapy community.☆560Dec 28, 2022Updated 3 years ago
- Python clients for Zyte AutoExtract API☆41Jan 17, 2022Updated 4 years ago
- GridSound wants to be a free browser-based HTML5 DAW (Digital Audio Workstation) following the new Web Audio API. You can test the applic…☆12Dec 9, 2018Updated 7 years ago
- Extract text from HTML☆135Apr 8, 2026Updated 3 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Web scraping Page Objects core library☆107Jun 22, 2026Updated 2 weeks ago
- ☆14Jul 3, 2026Updated last week
- Apify actor to run web spiders written in Python in the Scrapy library☆13Dec 11, 2022Updated 3 years ago
- A Scrapy extension to log items coverage when the spider shuts down☆18Apr 11, 2020Updated 6 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆45Mar 26, 2021Updated 5 years ago
- Automatic unit test generation for Scrapy.☆58Jul 12, 2021Updated 5 years ago
- Python interface to Digital Ocean☆25Jun 11, 2015Updated 11 years ago
- Zyte Automatic Extraction integration for Scrapy☆58Apr 13, 2026Updated 2 months ago
- Page Object pattern for Scrapy☆127Jun 8, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Example code for my SunshinePHP Guzzle Tutorial☆10Feb 5, 2015Updated 11 years ago
- humanreadable is a Python library to convert human-readable values to other units.☆21Jun 22, 2026Updated 2 weeks ago
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website☆43Oct 13, 2017Updated 8 years ago
- HTML5 audio/video clipper☆12Mar 7, 2018Updated 8 years ago
- Convert Javascript code to an XML document☆188Mar 14, 2022Updated 4 years ago
- Common interface for data container classes☆70May 6, 2026Updated 2 months ago
- A scrapy middleware to save http cache to MongoDB☆13Apr 13, 2026Updated 2 months ago
- Detect and classify pagination links☆107Apr 8, 2026Updated 3 months ago
- Extract price amount and currency symbol from a raw text string☆346Mar 19, 2026Updated 3 months ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆10Apr 22, 2024Updated 2 years ago
- Helped during my OSCP lab days.☆16Dec 7, 2016Updated 9 years ago
- Scrapy spider middleware to split an item into multiple items using a multi-valued key☆21Feb 8, 2017Updated 9 years ago
- Auto generated swig python module with a binary compnent☆11Apr 19, 2012Updated 14 years ago
- Scrapy project template. Use it to quickly spin up a new web scraping project☆16Nov 18, 2024Updated last year
- ☆10Jun 23, 2026Updated 2 weeks ago
- Podclips is an iOS app that allows users to cut out and share clips from their favourite podcasts☆15Mar 25, 2018Updated 8 years ago
- OnionSprout is a tool to run publicaly-accessible web services, for example from Raspberry Pi in your home, without a public IP.☆11Oct 3, 2020Updated 5 years ago
- Unofficial API to fetch bin details from bins.ws☆15Jul 9, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A list of Github users who distribute tools and tutorials on OSINT (as well as hacking, pentest, and forensics)☆17Oct 15, 2023Updated 2 years ago
- A simple, Qt-Webengine powered web browser with built in functionality for basic scrapy webscraping support.☆109May 21, 2024Updated 2 years ago
- Digital Ocean API Dashboard☆18Nov 2, 2015Updated 10 years ago
- Senarai padanan kata dan istilah populer.☆10Jun 21, 2017Updated 9 years ago
- A linter for Scrapy projects.☆22Updated this week
- 批量生成http的user-agent；多设备、多系统、多机型；自定义参数和概率☆15Mar 19, 2019Updated 7 years ago
- Nim wrapper for librdkafka☆10Dec 28, 2023Updated 2 years ago