A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
☆60May 10, 2026Updated 2 weeks ago
Alternatives and similar repositories for scrapy-distributed
Users that are interested in scrapy-distributed are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Type-safe CLI argument parser☆11Jul 29, 2024Updated last year
- PHP Magic Constants, even more powerful than an Enum☆13Jan 26, 2026Updated 3 months ago
- ☆25Mar 31, 2026Updated last month
- Implement scrapy with asyncio☆71Sep 4, 2025Updated 8 months ago
- Run selenium undetected.☆32Mar 6, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- More flexible and featured Frontera scheduler for Scrapy☆36Jun 6, 2025Updated 11 months ago
- pip install realtime_web_logs 文件日志实时显示到web页面。附带全系统硬盘的文件浏览下载功能。支持日志显示自动滚动和暂停。☆13Jul 1, 2020Updated 5 years ago
- Scrapyd on container infrastructure☆16Apr 11, 2025Updated last year
- PayU payment plugin for Sylius applications.☆25May 14, 2026Updated last week
- js_rpc通杀版本☆14Oct 9, 2021Updated 4 years ago
- Script for setting up ADB for debugging over wifi☆13Aug 12, 2019Updated 6 years ago
- ☆13May 11, 2022Updated 4 years ago
- Interactive Web Plotting with Bokeh in IPython notebook☆34Feb 25, 2018Updated 8 years ago
- A collection of pipelines for Scrapy☆16Apr 27, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- lightweight mvc framework written with gdscript.☆22Feb 27, 2025Updated last year
- pymysql 连接池☆14Mar 18, 2019Updated 7 years ago
- 🔌 PSR7 Worker Client for RoadRunner application server.☆80Feb 26, 2026Updated 2 months ago
- Generate Chinese name according to statistic model.☆13Sep 4, 2015Updated 10 years ago
- Python 业务开发常见错误案例集 配套源代码☆10Dec 19, 2020Updated 5 years ago
- ☆11Aug 26, 2018Updated 7 years ago
- Gossh is a remote login client that makes up for the inability of terminal ssh to save passwords. Gossh是一个远程登录客户端，它解决了终端ssh无法保存密码的问题。☆16Apr 18, 2018Updated 8 years ago
- 基于puppeteer和NodeJS的服务端渲染，提供Docker一键部署及API调用接口。☆19Aug 30, 2022Updated 3 years ago
- Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js☆3,501Oct 29, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Kubernetes KMS provider plugin, backed by Tencent Cloud KMS service☆17Jul 25, 2024Updated last year
- 通过DBUtils库实现Django数据库连接池，目前支持mysql、postgresql 数据库.☆12Jun 10, 2021Updated 4 years ago
- Downloader Middleware to support Playwright in Scrapy & Gerapy☆111Mar 6, 2022Updated 4 years ago
- 易盾图标识别，包含定位以及点选顺序的识别，定位 pytorch 模型大小只有3M，执行速度极快。内附代码和测试用例，直接使用即可测试。定位准确率 95% 以上，识别用的sift算法，测试通过率大概 50%。☆32Jun 2, 2020Updated 5 years ago
- Scrapy + Puppeteer☆110Jun 11, 2021Updated 4 years ago
- IRC server written in Rust☆20Nov 26, 2014Updated 11 years ago
- 一个简易且可扩展的消息通 知组件，已内置钉钉，企微，bark，pushover，pushdeer，飞书，邮件等消息通知组件☆27Mar 15, 2026Updated 2 months ago
- ☆14Dec 8, 2023Updated 2 years ago
- 关于5000+站点的scrapy爬虫开发，涉及一些技术架构搭建以及各种反爬方案，详见readme文件☆30Dec 8, 2022Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- 🔥 PHP library to warm up caches of URLs located in XML sitemaps☆74Updated this week
- Adds Flyway as an build step in Jenkins☆15May 12, 2026Updated last week
- PyTest Salt Plugin☆13Jan 27, 2020Updated 6 years ago
- SonataTranslationBundle☆77May 3, 2026Updated 3 weeks ago
- User Agent Switcher + for Chrome☆11Apr 9, 2022Updated 4 years ago
- javascript逆向开发基础组件，监听window的变化☆18Feb 11, 2024Updated 2 years ago
- Get CSS path to an element.☆18Dec 15, 2012Updated 13 years ago