tenlee2012/scrapy-kafka-redis external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

tenlee2012/scrapy-kafka-redis

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/tenlee2012/scrapy-kafka-redis)

Close

tenlee2012 / scrapy-kafka-redisView on GitHubMore

• External links
• Readme badge

Distributed crawling/scraping, Kafka And Redis based components for Scrapy

☆46Nov 13, 2020Updated 5 years ago

Alternatives and similar repositories for scrapy-kafka-redis

Users that are interested in scrapy-kafka-redis are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• dfdeshom / scrapy-kafka

  View on GitHub
  Kafka-based components for Scrapy
  ☆78Apr 10, 2018Updated 8 years ago
• AaronJny / scrapy_redis_expiredupefilter

  View on GitHub
  scrapy-redis-expiredupefilter是基于scrapy-redis修改来的一款scrapy分布式爬虫框架，它支持为请求指纹设置生命周期，请求指纹生命周期结束后将在不影响其他指纹的情况下自动清除。
  ☆10Aug 6, 2019Updated 6 years ago
• sunhailin-Leo / Scrapy-Kafka-Demo

  View on GitHub
  Scrapy and Kafka
  ☆14Feb 7, 2018Updated 8 years ago
• spicyparrot / kafka_scrapy_connect

  View on GitHub
  A custom library that integrates Scrapy with Kafka.
  ☆12Apr 27, 2026Updated last week
• locoz666 / switch-cloud

  View on GitHub
  Nintendo Switch 云游戏！
  ☆12May 8, 2020Updated 6 years ago
• Deploy on Railway without the complexity - Free Credits Offer • Ad
  Connect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
• jinuljt / wechat-web-auth

  View on GitHub
  对微信网页授权获取用户信息的封装
  ☆10Jul 30, 2015Updated 10 years ago
• Masutangu / SuperScripter

  View on GitHub
  A Simple Tool to Distribute/Administrate Your Scripts
  ☆12Sep 10, 2015Updated 10 years ago
• Germey / AdslProxy

  View on GitHub
  ☆17Jul 14, 2017Updated 8 years ago
• LiuXingMing / Scrapy_Redis_Bloomfilter

  View on GitHub
  基于Redis的Bloomfilter去重，并将其扩展到Scrapy框架。
  ☆346Feb 26, 2023Updated 3 years ago
• huichen / batchqueue

  View on GitHub
  批处理延迟任务队列
  ☆53Aug 8, 2013Updated 12 years ago
• Gerapy / GerapyAutoExtractor

  View on GitHub
  Auto Extractor Module
  ☆334Aug 19, 2024Updated last year
• clemfromspace / scrapy-puppeteer

  View on GitHub
  Scrapy + Puppeteer
  ☆110Jun 11, 2021Updated 4 years ago
• aylhex / book-code

  View on GitHub
  《App安全实战指南：Android和iOS App的安全攻防与合规 》书中的示例代码和相关工具
  ☆40Sep 2, 2024Updated last year
• BruceDone / clock

  View on GitHub
  可视化任务调度系统，精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)
  ☆195Mar 21, 2026Updated last month
• End-to-end encrypted email - Proton Mail • Ad
  Special offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
• iMartyn / helm-samba4

  View on GitHub
  Helm Chart for Samba4
  ☆10Oct 13, 2021Updated 4 years ago
• michellebonat / tf_text_classify

  View on GitHub
  The classic movies redux with machine learning using TensorFlow and Keras.
  ☆11Feb 12, 2019Updated 7 years ago
• CrazyLittleArmy / DocumentPreview

  View on GitHub
  一款优秀的在线文件预览解决方案，，使用主流springboot+maven搭建，支持doc、docx、ppt、pptx、xls、xlsx、zip、rar、mp4、mp3以及众多类文本如txt、html、xml、java、properties、sql、js、md、json、c…
  ☆16Dec 6, 2022Updated 3 years ago
• goplus / hdq

  View on GitHub
  HTML DOM Query Language for XGo
  ☆42Feb 16, 2026Updated 2 months ago
• Cospel / kaggle-rossmann

  View on GitHub
  Prediction model for Kaggle/Rossmann competition.
  ☆13Nov 23, 2015Updated 10 years ago
• idealspark / ZhiShiWenDa-Helper

  View on GitHub
  百万英雄/冲顶大会/知识超人 答题助手 瞬间使用Chrome打开百度
  ☆101Jan 21, 2018Updated 8 years ago
• zkqiang / job-spider

  View on GitHub
  多线程爬取互联网行业常用招聘网站
  ☆29Mar 4, 2018Updated 8 years ago
• xiaxichen / zh_login

  View on GitHub
  知乎登录
  ☆22Mar 18, 2019Updated 7 years ago
• Tioit-Wang / sanic-rest-framework

  View on GitHub
  API rapid development framework for SANIC, Inspired by Django REST Framework.
  ☆11Aug 29, 2025Updated 8 months ago
• Serverless GPU API endpoints on Runpod - Get Bonus Credits • Ad
  Skip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
• elacuesta / scrapy-pyppeteer

  View on GitHub
  Pyppeteer integration for Scrapy
  ☆58Feb 26, 2021Updated 5 years ago
• GeneralNewsExtractor / GneList

  View on GitHub
  A chrome extension to get XPath of list items in webpage easily.
  ☆34Mar 11, 2022Updated 4 years ago
• Python3WebSpider / ScrapyPyppeteerDeprecated

  View on GitHub
  Scrapy Pyppeteer Demo
  ☆24Jul 13, 2018Updated 7 years ago
• Python3WebSpider / ScrapyUniversal

  View on GitHub
  Scrapy Universal Spider
  ☆57Aug 26, 2017Updated 8 years ago
• siseng / siseng.github.io

  View on GitHub
  homepage
  ☆10Feb 15, 2023Updated 3 years ago
• billxc / feishu-gpt-python

  View on GitHub
  基于Azure OpenAI的飞书机器人
  ☆13Apr 18, 2023Updated 3 years ago
• oldsyang / weixin_pay

  View on GitHub
  ☆12Dec 23, 2017Updated 8 years ago
• kingname / EverFlowy

  View on GitHub
  Sync your workflowy's items into evernote
  ☆32Dec 8, 2022Updated 3 years ago
• locoz666 / gamepad-agent

  View on GitHub
  一个纯 Go 实现的游戏手柄网络转发工具（实际上用的库并不纯）
  ☆13May 8, 2020Updated 6 years ago
• GPU virtual machines on DigitalOcean Gradient AI • Ad
  Get to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
• fengxiaochuang / ScrapyDemo

  View on GitHub
  ScrapyDemo : Redis MySQLdb logging IngoreHttpRequestMiddleware UserAgentMiddleware HttpProxyMiddleware rules
  ☆38Jun 28, 2016Updated 9 years ago
• bytebuff / aioScrapy

  View on GitHub
  基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star
  ☆35Oct 25, 2019Updated 6 years ago
• public-accountability / littlesis-main

  View on GitHub
  LittleSis development setup w/ docker & deployment instructions w/ ansible
  ☆13Jan 7, 2022Updated 4 years ago
• zhonghangAlex / EncryptOpDemo

  View on GitHub
  这是一个基于js的加密解密算法项目
  ☆13Nov 18, 2022Updated 3 years ago
• yuweiping / WeChatOpenDevTools

  View on GitHub
  ☆12Apr 4, 2024Updated 2 years ago
• Kirk-Wang / Hello-Gopher

  View on GitHub
  Go Programming Language 扫盲
  ☆12Sep 7, 2020Updated 5 years ago
• smallnest / mcp-prompt-server-go

  View on GitHub
  一个提供优秀prompt的Model Context Protocol (MCP)的服务器，用于根据用户任务需求提供预设的prompt模板，帮助Cline/Cursor/Windsurf...更高效地执行各种任务。服务器将预设的prompt作为工具(tools)返回，以便在…
  ☆34May 21, 2025Updated 11 months ago