tenlee2012/scrapy-kafka-redis external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

tenlee2012/scrapy-kafka-redis

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/tenlee2012/scrapy-kafka-redis)

Close

tenlee2012 / scrapy-kafka-redisView on GitHubMore

• External links
• Readme badge

Distributed crawling/scraping, Kafka And Redis based components for Scrapy

☆46Nov 13, 2020Updated 5 years ago

Alternatives and similar repositories for scrapy-kafka-redis

Users that are interested in scrapy-kafka-redis are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• dfdeshom / scrapy-kafka

  View on GitHub
  Kafka-based components for Scrapy
  ☆78Apr 10, 2018Updated 8 years ago
• AaronJny / scrapy_redis_expiredupefilter

  View on GitHub
  scrapy-redis-expiredupefilter是基于scrapy-redis修改来的一款scrapy分布式爬虫框架，它支持为请求指纹设置生命周期，请求指纹生命周期结束后将在不影响其他指纹的情况下自动清除。
  ☆10Aug 6, 2019Updated 6 years ago
• sunhailin-Leo / Scrapy-Kafka-Demo

  View on GitHub
  Scrapy and Kafka
  ☆14Feb 7, 2018Updated 8 years ago
• OneJane / OneJane.github.io

  View on GitHub
  个人博客
  ☆13Feb 2, 2023Updated 3 years ago
• locoz666 / switch-cloud

  View on GitHub
  Nintendo Switch 云游戏！
  ☆12May 8, 2020Updated 5 years ago
• Managed hosting for WordPress and PHP on Cloudways • Ad
  Managed hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
• arkady-emelyanov / kafkabeat

  View on GitHub
  Kafka event forwarder build on top of Elastic Beats platform
  ☆12Apr 29, 2019Updated 6 years ago
• roycehaynes / scrapy-rabbitmq

  View on GitHub
  A RabbitMQ Scheduler for Scrapy
  ☆87Aug 9, 2022Updated 3 years ago
• jinuljt / wechat-web-auth

  View on GitHub
  对微信网页授权获取用户信息的封装
  ☆10Jul 30, 2015Updated 10 years ago
• panghaibin / httpcanary_spider

  View on GitHub
  一个基于 HttpCanary 和 Python 的爬虫项目
  ☆21May 2, 2023Updated 2 years ago
• sosedoff / nginx2influxdb

  View on GitHub
  Stream Nginx logs directly into InfluxDB
  ☆14Sep 22, 2017Updated 8 years ago
• Germey / AdslProxy

  View on GitHub
  ☆17Jul 14, 2017Updated 8 years ago
• LiuXingMing / Scrapy_Redis_Bloomfilter

  View on GitHub
  基于Redis的Bloomfilter去重，并将其扩展到Scrapy框架。
  ☆346Feb 26, 2023Updated 3 years ago
• Gerapy / GerapyAutoExtractor

  View on GitHub
  Auto Extractor Module
  ☆334Aug 19, 2024Updated last year
• aylhex / book-code

  View on GitHub
  《App安全实战指南：Android和iOS App的安全攻防与合规 》书中的示例代码和相关工具
  ☆39Sep 2, 2024Updated last year
• GPU virtual machines on DigitalOcean Gradient AI • Ad
  Get to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
• BruceDone / clock

  View on GitHub
  可视化任务调度系统，精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)
  ☆194Mar 21, 2026Updated 3 weeks ago
• JeffreyDing11223 / kago

  View on GitHub
  the Go client to use Kafka . Based on sarama and sarama-cluster.
  ☆14Aug 10, 2018Updated 7 years ago
• iMartyn / helm-samba4

  View on GitHub
  Helm Chart for Samba4
  ☆10Oct 13, 2021Updated 4 years ago
• r1is / xiaolanben_h_sign

  View on GitHub
  小蓝本(https://www.xiaolanben.com/) 爬虫的 h_sign 签名JSRPC实现。nodejs 补环境也实现了
  ☆13Apr 30, 2024Updated last year
• Cospel / kaggle-rossmann

  View on GitHub
  Prediction model for Kaggle/Rossmann competition.
  ☆13Nov 23, 2015Updated 10 years ago
• idealspark / ZhiShiWenDa-Helper

  View on GitHub
  百万英雄/冲顶大会/知识超人 答题助手 瞬间使用Chrome打开百度
  ☆101Jan 21, 2018Updated 8 years ago
• asyncins / antispider

  View on GitHub
  书籍《Python3 反爬虫原理与绕过实战》配套代码
  ☆628Oct 25, 2021Updated 4 years ago
• zkqiang / job-spider

  View on GitHub
  多线程爬取互联网行业常用招聘网站
  ☆29Mar 4, 2018Updated 8 years ago
• xiaxichen / zh_login

  View on GitHub
  知乎登录
  ☆22Mar 18, 2019Updated 7 years ago
• Simple, predictable pricing with DigitalOcean hosting • Ad
  Always know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
• istresearch / scrapy-cluster

  View on GitHub
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,228Nov 7, 2023Updated 2 years ago
• elacuesta / scrapy-pyppeteer

  View on GitHub
  Pyppeteer integration for Scrapy
  ☆58Feb 26, 2021Updated 5 years ago
• GeneralNewsExtractor / GneList

  View on GitHub
  A chrome extension to get XPath of list items in webpage easily.
  ☆35Mar 11, 2022Updated 4 years ago
• Python3WebSpider / ScrapyUniversal

  View on GitHub
  Scrapy Universal Spider
  ☆58Aug 26, 2017Updated 8 years ago
• siseng / siseng.github.io

  View on GitHub
  homepage
  ☆10Feb 15, 2023Updated 3 years ago
• billxc / feishu-gpt-python

  View on GitHub
  基于Azure OpenAI的飞书机器人
  ☆13Apr 18, 2023Updated 3 years ago
• HanEightTurtle / mitm_server_ql

  View on GitHub
  ☆11Mar 16, 2022Updated 4 years ago
• kingname / EverFlowy

  View on GitHub
  Sync your workflowy's items into evernote
  ☆32Dec 8, 2022Updated 3 years ago
• fengxiaochuang / ScrapyDemo

  View on GitHub
  ScrapyDemo : Redis MySQLdb logging IngoreHttpRequestMiddleware UserAgentMiddleware HttpProxyMiddleware rules
  ☆38Jun 28, 2016Updated 9 years ago
• Wordpress hosting with auto-scaling - Free Trial • Ad
  Fully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
• locoz666 / gamepad-agent

  View on GitHub
  一个纯 Go 实现的游戏手柄网络转发工具（实际上用的库并不纯）
  ☆13May 8, 2020Updated 5 years ago
• bytebuff / aioScrapy

  View on GitHub
  基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star
  ☆35Oct 25, 2019Updated 6 years ago
• DansProjects / airflow-averageface

  View on GitHub
  Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website
  ☆44Oct 13, 2017Updated 8 years ago
• zhonghangAlex / EncryptOpDemo

  View on GitHub
  这是一个基于js的加密解密算法项目
  ☆13Nov 18, 2022Updated 3 years ago
• yuweiping / WeChatOpenDevTools

  View on GitHub
  ☆12Apr 4, 2024Updated 2 years ago
• Kirk-Wang / Hello-Gopher

  View on GitHub
  Go Programming Language 扫盲
  ☆12Sep 7, 2020Updated 5 years ago
• Neoteroi / BlackSheep-SQLAlchemy

  View on GitHub
  Extension for BlackSheep that simplifies the use of SQLAlchemy in the web framework.
  ☆16Mar 27, 2025Updated last year