πΆ Awesome list of Scrapy tools and libraries
β61Jul 6, 2020Updated 6 years ago
Alternatives and similar repositories for awesome-scrapy
Users that are interested in awesome-scrapy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A curated list of awesome packages, articles, and other cool resources from the Scrapy community.β563Dec 28, 2022Updated 3 years ago
- Library to populate items using XPath and CSS with a convenient APIβ49Nov 1, 2026Updated 2 weeks ago
- A library to make it easier to load input URLs to start scrapy processesβ14Feb 21, 2021Updated 5 years ago
- Pluggable DSL that uses pipes to perform a series of linear transformations to extract dataβ16Jul 9, 2024Updated 2 years ago
- Python clients for Zyte AutoExtract APIβ41Jan 17, 2022Updated 4 years ago
- Simple, predictable pricing with DigitalOcean hosting β’ AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Scrapy project template. Use it to quickly spin up a new web scraping projectβ16Nov 18, 2024Updated 2 years ago
- A list of awesome project for Ruiaβ13Aug 24, 2022Updated 4 years ago
- β10Oct 22, 2018Updated 8 years ago
- GridSound wants to be a free browser-based HTML5 DAW (Digital Audio Workstation) following the new Web Audio API. You can test the applicβ¦β12Dec 9, 2018Updated 7 years ago
- A browser extension to monitor your spiders deployed on Scrapy Cloud.β16Mar 8, 2025Updated last year
- Web scraping Page Objects core libraryβ107Nov 12, 2026Updated last week
- An MCP server that provides real-time data and insights from the Hyperliquid perp DEX for use in bots, dashboards, and analytics.β30May 31, 2025Updated last year
- Library for annotation-based dependency injectionβ24Nov 1, 2026Updated 2 weeks ago
- A Scrapy extension to log items coverage when the spider shuts downβ18Apr 11, 2020Updated 6 years ago
- GPU virtual machines on DigitalOcean Gradient AI β’ AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Common interface for data container classesβ71Nov 13, 2026Updated last week
- Automatic unit test generation for Scrapy.β58Jul 12, 2021Updated 5 years ago
- Zyte Automatic Extraction integration for Scrapyβ58Apr 13, 2026Updated 7 months ago
- Scrapy project boilerplate done rightβ49Feb 26, 2026Updated 8 months ago
- An MCP server that provides real-time whale alerts on Hyperliquid.β13Nov 16, 2025Updated last year
- π Playwright integration for Scrapyβ1,470Nov 2, 2026Updated 2 weeks ago
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given websiteβ43Oct 13, 2017Updated 9 years ago
- humanreadable is a Python library to convert human-readable values to other units.β21Jul 20, 2026Updated 4 months ago
- MCP Hyperliquid (https://app.hyperliquid.xyz) serverβ44Mar 6, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean β’ AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.β¦β3,411Feb 19, 2025Updated last year
- Scrapy Extension for monitoring spiders execution.β564Oct 20, 2026Updated last month
- Contains the common item definitions used in Zyte.β10Nov 1, 2026Updated 2 weeks ago
- A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.β95Jan 5, 2025Updated last year
- Convert Javascript code to an XML documentβ186Mar 14, 2022Updated 4 years ago
- A decorator to write coroutine-like spider callbacks.β108Dec 26, 2022Updated 3 years ago
- More flexible and featured Frontera scheduler for Scrapyβ38Sep 17, 2026Updated 2 months ago
- Helped during my OSCP lab days.β17Dec 7, 2016Updated 9 years ago
- Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.jsβ3,518Jul 4, 2026Updated 4 months ago
- Deploy on Railway without the complexity - Free Credits Offer β’ AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Extract price amount and currency symbol from a raw text stringβ351Nov 2, 2026Updated 2 weeks ago
- OnionSprout is a tool to run publicaly-accessible web services, for example from Raspberry Pi in your home, without a public IP.β12Oct 3, 2020Updated 6 years ago
- Detect and classify pagination linksβ107Apr 8, 2026Updated 7 months ago
- ansible-playbook. Add and configure zabbix-agent.β13Sep 18, 2023Updated 3 years ago
- A collection of awesome web scaper, crawler.β296Updated this week
- A simple, Qt-Webengine powered web browser with built in functionality for basic scrapy webscraping support.β110May 21, 2024Updated 2 years ago
- scrapydweb, dockerfileβ13Feb 1, 2021Updated 5 years ago
Are these results useful?