A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
☆62May 10, 2026Updated 3 months ago
Alternatives and similar repositories for scrapy-distributed
Users that are interested in scrapy-distributed are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Type-safe CLI argument parser☆11Jul 29, 2024Updated 2 years ago
- ☆25Jun 24, 2026Updated 2 months ago
- Implement scrapy with asyncio☆71Jul 23, 2026Updated last month
- pip install realtime_web_logs 文件日志实时显示到web页面。附带全系统硬盘的文件浏览下载功能。支持日志显示自动滚动和暂停。☆13Jul 1, 2020Updated 6 years ago
- PayU payment plugin for Sylius applications.☆25Jun 30, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆13May 11, 2022Updated 4 years ago
- A plugin for Oobabooga TextUI that allows you to search multiple search engines. Initially we're using Google API or DuckDuckGo.☆18Jun 4, 2023Updated 3 years ago
- ☆15Jan 22, 2021Updated 5 years ago
- Simple token based authentication for Django☆12Sep 11, 2023Updated 2 years ago
- This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.☆1,226Nov 7, 2023Updated 2 years ago
- 使用mnn-llm对GOT-OCR2.0进行推理☆13Oct 2, 2024Updated last year
- ☆21Sep 28, 2024Updated last year
- Quickly deploy Graylog 2.x in Rancher 1.6x☆10Nov 14, 2021Updated 4 years ago
- Python 业务开发常见错误案例集 配套源代码☆10Dec 19, 2020Updated 5 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- 基于puppeteer和NodeJS的服务端渲染，提供Docker一键部署及API调用接口。☆19Aug 30, 2022Updated 4 years ago
- Downloader Middleware to support Playwright in Scrapy & Gerapy☆111Mar 6, 2022Updated 4 years ago
- 易盾图标识别，包含定位以及点选顺序的识别，定位 pytorch 模型大小只有3M，执行速度极快。内附代码和测试用例，直接使用即可测试。定位准确率 95% 以上，识别用的sift算法，测试通过率大概 50%。☆34Jun 2, 2020Updated 6 years ago
- Fostering connections in Singapore☆16Oct 18, 2016Updated 9 years ago
- 多端抓取抖音用户数据☆32May 13, 2019Updated 7 years ago
- 🔥 PHP library to warm up caches of URLs located in XML sitemaps☆77Updated this week
- 关于5000+站点的scrapy爬虫开发，涉及一些技术架构搭建以及各种反爬方案，详见readme文件☆31Dec 8, 2022Updated 3 years ago
- Scrapy + Puppeteer☆110Jun 11, 2021Updated 5 years ago
- SonataTranslationBundle☆77May 3, 2026Updated 4 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Expiring token with expiration time update for Django Rest Framework☆10Feb 4, 2024Updated 2 years ago
- JMeter Tester with Influxdb and Grafana☆14Apr 10, 2020Updated 6 years ago
- Scrapy Pyppeteer Demo☆12Jul 30, 2020Updated 6 years ago
- gitchat课程下载工具☆30Jul 11, 2020Updated 6 years ago
- 这是一个基于 FastAPI 的 浏览器 RPC 服务端☆59Aug 17, 2023Updated 3 years ago
- Scrapy GUI☆12Feb 26, 2021Updated 5 years ago
- Live templates for PHPStorm and Laravel☆16Jun 7, 2014Updated 12 years ago
- User Agent Switcher + for Chrome☆11Apr 9, 2022Updated 4 years ago
- Tools for Crawlers☆22Dec 25, 2023Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Support view/rendering of Laravel pages and templates from a database.☆12Nov 22, 2017Updated 8 years ago
- Easily implement distributed asynchronous tasks in one step.仅需一步，轻松实现分布式异步任务。☆23Updated this week
- Trace function calls using lldb☆13Jul 5, 2021Updated 5 years ago
- laravel5 multi auth☆11Mar 4, 2015Updated 11 years ago
- Broad crawler for domain discovery☆20Apr 8, 2026Updated 4 months ago
- 作文网全站内容 EPUB 备份☆13Oct 4, 2023Updated 2 years ago
- A consul-backed configuration source for Archaius☆10Nov 16, 2015Updated 10 years ago