tenlee2012/scrapy-kafka-redis external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

tenlee2012/scrapy-kafka-redis

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/tenlee2012/scrapy-kafka-redis)

Close

tenlee2012 / scrapy-kafka-redisView on GitHubMore

• External links
• Readme badge

Distributed crawling/scraping, Kafka And Redis based components for Scrapy

☆46Nov 13, 2020Updated 5 years ago

Alternatives and similar repositories for scrapy-kafka-redis

Users that are interested in scrapy-kafka-redis are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• sunhailin-Leo / Scrapy-Kafka-Demo

  View on GitHub
  Scrapy and Kafka
  ☆14Feb 7, 2018Updated 8 years ago
• spicyparrot / kafka_scrapy_connect

  View on GitHub
  A custom library that integrates Scrapy with Kafka.
  ☆12Apr 27, 2026Updated last month
• OneJane / OneJane.github.io

  View on GitHub
  个人博客
  ☆13Feb 2, 2023Updated 3 years ago
• arkady-emelyanov / kafkabeat

  View on GitHub
  Kafka event forwarder build on top of Elastic Beats platform
  ☆12Apr 29, 2019Updated 7 years ago
• samuelyao314 / nsqbook

  View on GitHub
  nsq官方文档翻译和源码学习
  ☆11Sep 10, 2014Updated 11 years ago
• End-to-end encrypted cloud storage - Proton Drive • Ad
  Special offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
• roycehaynes / scrapy-rabbitmq

  View on GitHub
  A RabbitMQ Scheduler for Scrapy
  ☆86Aug 9, 2022Updated 3 years ago
• jinuljt / wechat-web-auth

  View on GitHub
  对微信网页授权获取用户信息的封装
  ☆10Jul 30, 2015Updated 10 years ago
• panghaibin / httpcanary_spider

  View on GitHub
  一个基于 HttpCanary 和 Python 的爬虫项目
  ☆21May 2, 2023Updated 3 years ago
• sosedoff / nginx2influxdb

  View on GitHub
  Stream Nginx logs directly into InfluxDB
  ☆14Sep 22, 2017Updated 8 years ago
• Germey / AdslProxy

  View on GitHub
  ☆17Jul 14, 2017Updated 8 years ago
• LiuXingMing / Scrapy_Redis_Bloomfilter

  View on GitHub
  基于Redis的Bloomfilter去重，并将其扩展到Scrapy框架。
  ☆346Feb 26, 2023Updated 3 years ago
• freedom-wy / reverse_android

  View on GitHub
  安卓从开发到逆向
  ☆79Oct 6, 2019Updated 6 years ago
• Gerapy / GerapyAutoExtractor

  View on GitHub
  Auto Extractor Module
  ☆334Aug 19, 2024Updated last year
• clemfromspace / scrapy-puppeteer

  View on GitHub
  Scrapy + Puppeteer
  ☆110Jun 11, 2021Updated 4 years ago
• Wordpress hosting with auto-scaling - Free Trial Offer • Ad
  Fully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
• BruceDone / clock

  View on GitHub
  可视化任务调度系统，精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)
  ☆195Mar 21, 2026Updated 2 months ago
• iMartyn / helm-samba4

  View on GitHub
  Helm Chart for Samba4
  ☆10Oct 13, 2021Updated 4 years ago
• dagger / hello-dagger

  View on GitHub
  Dagger Quickstart - Example Application
  ☆10May 21, 2026Updated last week
• CrazyLittleArmy / DocumentPreview

  View on GitHub
  一款优秀的在线文件预览解决方案，，使用主流springboot+maven搭建，支持doc、docx、ppt、pptx、xls、xlsx、zip、rar、mp4、mp3以及众多类文本如txt、html、xml、java、properties、sql、js、md、json、c…
  ☆16Dec 6, 2022Updated 3 years ago
• r1is / xiaolanben_h_sign

  View on GitHub
  小蓝本(https://www.xiaolanben.com/) 爬虫的 h_sign 签名JSRPC实现。nodejs 补环境也实现了
  ☆14Apr 30, 2024Updated 2 years ago
• goplus / hdq

  View on GitHub
  HTML DOM Query Language for XGo
  ☆42Feb 16, 2026Updated 3 months ago
• kingname / TeamFlowy

  View on GitHub
  A simple sync tool to sync task from Workflowy to Teambition
  ☆32Oct 4, 2017Updated 8 years ago
• asyncins / antispider

  View on GitHub
  书籍《Python3 反爬虫原理与绕过实战》配套代码
  ☆628Oct 25, 2021Updated 4 years ago
• xiaxichen / zh_login

  View on GitHub
  知乎登录
  ☆22Mar 18, 2019Updated 7 years ago
• Deploy on Railway without the complexity - Free Credits Offer • Ad
  Connect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
• istresearch / scrapy-cluster

  View on GitHub
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,226Nov 7, 2023Updated 2 years ago
• lixi5338619 / r0capture

  View on GitHub
  安卓应用层抓包通杀脚本
  ☆10Jan 4, 2021Updated 5 years ago
• Tioit-Wang / sanic-rest-framework

  View on GitHub
  API rapid development framework for SANIC, Inspired by Django REST Framework.
  ☆11Aug 29, 2025Updated 9 months ago
• elacuesta / scrapy-pyppeteer

  View on GitHub
  Pyppeteer integration for Scrapy
  ☆58Feb 26, 2021Updated 5 years ago
• GeneralNewsExtractor / GneList

  View on GitHub
  A chrome extension to get XPath of list items in webpage easily.
  ☆34Mar 11, 2022Updated 4 years ago
• Python3WebSpider / ScrapyUniversal

  View on GitHub
  Scrapy Universal Spider
  ☆57Aug 26, 2017Updated 8 years ago
• pahrohfit / sanic-beskar

  View on GitHub
  Strong, Simple, and Precise, (and now async!) security for Sanic APIs
  ☆14Jul 23, 2024Updated last year
• siseng / siseng.github.io

  View on GitHub
  homepage
  ☆10Feb 15, 2023Updated 3 years ago
• billxc / feishu-gpt-python

  View on GitHub
  基于Azure OpenAI的飞书机器人
  ☆13Apr 18, 2023Updated 3 years ago
• AI Agents on DigitalOcean Gradient AI Platform • Ad
  Build production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
• oldsyang / weixin_pay

  View on GitHub
  ☆11Dec 23, 2017Updated 8 years ago
• HanEightTurtle / mitm_server_ql

  View on GitHub
  ☆11Mar 16, 2022Updated 4 years ago
• kingname / EverFlowy

  View on GitHub
  Sync your workflowy's items into evernote
  ☆32Dec 8, 2022Updated 3 years ago
• fengxiaochuang / ScrapyDemo

  View on GitHub
  ScrapyDemo : Redis MySQLdb logging IngoreHttpRequestMiddleware UserAgentMiddleware HttpProxyMiddleware rules
  ☆38Jun 28, 2016Updated 9 years ago
• leokwsw / OpenAI-RefreshToken

  View on GitHub
  Use PandoraNext to Getn and Refresh ChatGPT Web Access Token
  ☆13Jan 19, 2024Updated 2 years ago
• bytebuff / aioScrapy

  View on GitHub
  基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star
  ☆35Oct 25, 2019Updated 6 years ago
• wenzhang-dev / bitcaskDB

  View on GitHub
  Light-weight, fast, fixed capacity key/value storage engine base on bitcask storage model
  ☆30Aug 25, 2025Updated 9 months ago