Alternatives and similar repositories for etlpy

Users that are interested in etlpy are comparing it to the libraries listed below

• ferventdesert / Hawk

  View on GitHub
  visualized crawler & ETL IDE written with C#/WPF
  ☆3,302Dec 21, 2019Updated 6 years ago
• FullerHua / gooseeker

  View on GitHub
  ☆694Oct 26, 2016Updated 9 years ago
• gnemoug / distribute_crawler

  View on GitHub
  使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
  ☆3,253Apr 18, 2017Updated 8 years ago
• binux / pyspider

  View on GitHub
  A Powerful Spider(Web Crawler) System in Python.
  ☆17,044Apr 30, 2024Updated last year
• qinxuye / cola

  View on GitHub
  A high-level distributed crawling framework.
  ☆1,506Jul 31, 2022Updated 3 years ago
• wuchong / scrapy-dynamic-configurable

  View on GitHub
  A dynamic configurable news crawler based Scrapy
  ☆164Jul 24, 2017Updated 8 years ago
• gangtao / dataplay2

  View on GitHub
  A simple data analysis software
  ☆284May 9, 2018Updated 7 years ago
• qiyeboy / IPProxyPool

  View on GitHub
  IPProxyPool代理池项目，提供代理ip
  ☆4,261Jul 13, 2018Updated 7 years ago
• Gerapy / Gerapy

  View on GitHub
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,500Oct 29, 2024Updated last year
• jackfengji / test_pro

  View on GitHub
  Doc for dpark which is a python rewrite for spark
  ☆154Feb 16, 2012Updated 13 years ago
• rmax / scrapy-redis

  View on GitHub
  Redis-based components for Scrapy.
  ☆5,648Jul 6, 2024Updated last year
• LiuXingMing / SinaSpider

  View on GitHub
  新浪微博爬虫（Scrapy、Redis）
  ☆3,280Sep 5, 2018Updated 7 years ago
• scrapinghub / portia

  View on GitHub
  Visual scraping for Scrapy
  ☆9,487Jun 26, 2024Updated last year
• xianhu / PSpider

  View on GitHub
  简单易用的Python爬虫框架，QQ交流群：597510560
  ☆1,836Jun 10, 2022Updated 3 years ago
• douban / dpark

  View on GitHub
  Python clone of Spark, a MapReduce alike framework in Python
  ☆2,675Dec 25, 2020Updated 5 years ago
• flyhigher139 / mayblog

  View on GitHub
  Yet another django based blog created in May 2015
  ☆163Jan 3, 2020Updated 6 years ago
• istresearch / scrapy-cluster

  View on GitHub
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,230Nov 7, 2023Updated 2 years ago
• chyroc / WechatSogou

  View on GitHub
  基于搜狗微信搜索的微信公众号爬虫接口
  ☆6,181Nov 15, 2023Updated 2 years ago
• dodng / indexbyi

  View on GitHub
  A tiny but complete full-text search http server (include make index,search engine,http server)
  ☆11May 2, 2016Updated 9 years ago
• ngs-doo / dsl-client-java

  View on GitHub
  DSL Platform - Java client
  ☆12Dec 10, 2016Updated 9 years ago
• buaazyc / Fast_faceswap

  View on GitHub
  fast_faceswap use dlib and change_style_network（基于dlib和风格迁移网络的快速换脸）
  ☆11Jul 18, 2019Updated 6 years ago
• waditu / tushare

  View on GitHub
  TuShare is a utility for crawling historical data of China stocks
  ☆14,424Mar 13, 2024Updated last year
• my8100 / scrapydweb

  View on GitHub
  Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.…
  ☆3,403Feb 19, 2025Updated 11 months ago
• aosabook / 500lines

  View on GitHub
  500 Lines or Less
  ☆29,607Aug 19, 2023Updated 2 years ago
• luyishisi / Anti-Anti-Spider

  View on GitHub
  越来越多的网站具有反爬虫特性，有的用图片隐藏关键数据，有的使用反人类的验证码，建立反反爬虫的代码仓库，通过与不同特性的网站做斗争（无恶意）提高技术。（欢迎提交难以采集的网站）（因工作原因，项目暂停）
  ☆7,301Oct 17, 2021Updated 4 years ago
• liujiannong / StockSenatus

  View on GitHub
  微信小程序，收集感兴趣的股票信息集中呈现，个人决策用。
  ☆11Dec 4, 2016Updated 9 years ago
• Arlenxiao / MetronicUI

  View on GitHub
  metronic魔方
  ☆10Dec 8, 2022Updated 3 years ago
• modulogrc / modSIC

  View on GitHub
  Modulo's Open Distributed SCAP Infrastructure Collector, or modSIC, makes it easier for security analysts to scan an environment vulnerab…
  ☆15Jun 23, 2017Updated 8 years ago
• codercaizh / springboot-dubbo

  View on GitHub
  springboot集成dubbo
  ☆11Jun 17, 2022Updated 3 years ago
• ning / Arecibo

  View on GitHub
  Real-time Monitoring
  ☆29May 14, 2012Updated 13 years ago
• dragnet-org / dragnet_data

  View on GitHub
  code and data used to build a training dataset for dragnet models
  ☆10Nov 29, 2020Updated 5 years ago
• jackgitgz / CnblogsSpider

  View on GitHub
  用scrapy采集cnblogs列表页爬虫
  ☆273Jun 16, 2015Updated 10 years ago
• grangier / python-goose

  View on GitHub
  Html Content / Article Extractor, web scrapping lib in Python
  ☆4,061Dec 26, 2021Updated 4 years ago
• yidao620c / core-scrapy

  View on GitHub
  python-scrapy demo
  ☆810Oct 1, 2020Updated 5 years ago
• Sssmeb / BloomFilter

  View on GitHub
  BloomFilter Based on py3(基于py3的布隆过滤器)
  ☆25Dec 7, 2022Updated 3 years ago
• dinp / builder

  View on GitHub
  DINP编配中心，把用户代码打包为Docker image
  ☆10Feb 8, 2015Updated 11 years ago
• zerosoul / strong-password-generator

  View on GitHub
  create strong passwords online
  ☆14Jan 31, 2026Updated 2 weeks ago
• BYVoid / byvnotes

  View on GitHub
  ☆26Apr 25, 2013Updated 12 years ago
• da2vin / Sasila

  View on GitHub
  一个灵活、友好的爬虫框架
  ☆297Jul 6, 2022Updated 3 years ago