Alternatives and similar repositories for scrapy_demo

Users that are interested in scrapy_demo are comparing it to the libraries listed below

• BruceDone / Tiktok

  View on GitHub
  Python web visualize build on the awesome web framework sanic
  ☆63Jul 12, 2017Updated 8 years ago
• BruceDone / dagobah

  View on GitHub
  Simple DAG-based job scheduler in Python
  ☆13May 10, 2017Updated 8 years ago
• BruceDone / cnbeta

  View on GitHub
  一键抓取cnbeta 首页的所有消息
  ☆16Sep 7, 2016Updated 9 years ago
• scrapy-plugins / scrapy-magicfields

  View on GitHub
  Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
  ☆57Mar 16, 2022Updated 3 years ago
• BruceDone / tensorflow-demos

  View on GitHub
  all kinds of demos of tensorflow code
  ☆95Oct 13, 2017Updated 8 years ago
• Python3WebSpider / ScrapySplashTest

  View on GitHub
  Scrapy Splash on Taobao Product
  ☆32Aug 6, 2017Updated 8 years ago
• xgkmxxx / Guazi

  View on GitHub
  scrapy抓取，mysql储存，django展示
  ☆12Feb 6, 2016Updated 10 years ago
• sbimochan / word-finder

  View on GitHub
  A CLI and Flask application that finds word from given letters.
  ☆16Sep 10, 2024Updated last year
• minms / ueditor-oss

  View on GitHub
  (JS+PHP) Ueditor 编辑器上传图片时, 直接上传至OSS, 不经过中间服务器(可配置回调)
  ☆15Aug 29, 2016Updated 9 years ago
• Gerapy / Gerapy

  View on GitHub
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,500Oct 29, 2024Updated last year
• a1phanumeric / csv2plist.py

  View on GitHub
  Convert a .csv to .plist array for usage with iOS / OSX app development
  ☆30Feb 17, 2013Updated 12 years ago
• wuchong / scrapy-dynamic-configurable

  View on GitHub
  A dynamic configurable news crawler based Scrapy
  ☆164Jul 24, 2017Updated 8 years ago
• groupbwt / scrapy-boilerplate

  View on GitHub
  Scrapy project boilerplate done right
  ☆48Feb 12, 2025Updated last year
• geekan / scrapy-examples

  View on GitHub
  Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
  ☆3,265Nov 3, 2023Updated 2 years ago
• libaibuaidufu / Fastapi-scheduler

  View on GitHub
  这是一个 fastapi 结合 apscheduler 做的一个动态添加定时任务的web
  ☆16Jan 14, 2022Updated 4 years ago
• AceSen / fast-exchange

  View on GitHub
  纯前端撸毛工具,OKX自动批量提币
  ☆19Oct 31, 2023Updated 2 years ago
• scrapy-plugins / scrapy-dotpersistence

  View on GitHub
  A scrapy extension to sync `.scrapy` folder to an S3 bucket
  ☆18Mar 28, 2022Updated 3 years ago
• shoma / mysqlmonitor

  View on GitHub
  MySQL Monitoring tools written in python
  ☆23Jul 4, 2014Updated 11 years ago
• Vespa314 / douban_scrapy

  View on GitHub
  将会陆续添加豆瓣里面各种信息的爬虫代码和分析
  ☆25Aug 11, 2014Updated 11 years ago
• DormyMo / SpiderKeeper

  View on GitHub
  admin ui for scrapy/open source scrapinghub
  ☆2,778May 4, 2023Updated 2 years ago
• istresearch / scrapy-cluster

  View on GitHub
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,230Nov 7, 2023Updated 2 years ago
• AndrewAndrea / spider_demo

  View on GitHub
  个人爬虫、逆向练习代码。文书web、app；facebook；毒；省市地区数据抓取，qcc 类似ob混淆, tb 滑动 demo，头条_signature等参数
  ☆55Feb 20, 2021Updated 4 years ago
• scrapinghub / scrapinghub-entrypoint-scrapy

  View on GitHub
  Scrapy entrypoint for Scrapinghub job runner
  ☆26Jan 28, 2026Updated 2 weeks ago
• rmax / scrapy-redis

  View on GitHub
  Redis-based components for Scrapy.
  ☆5,648Jul 6, 2024Updated last year
• ioiogoo / Proxies_

  View on GitHub
  一个简单的获取代理的类，代理来源http://www.xicidaili.com/
  ☆20Dec 13, 2016Updated 9 years ago
• scrapy-plugins / scrapy-pagestorage

  View on GitHub
  A scrapy extension to store requests and responses information in storage service
  ☆27Mar 11, 2022Updated 3 years ago
• Luniam / SimpleKVStore

  View on GitHub
  A distributed key value database based on LSM Tree storage
  ☆15Aug 24, 2022Updated 3 years ago
• djm / python-scrapyd-api

  View on GitHub
  A Python wrapper for working with Scrapyd's API.
  ☆271Jul 31, 2024Updated last year
• 360digitech / janusgraph-data-importer

  View on GitHub
  janusgraph data importer
  ☆24Jun 17, 2022Updated 3 years ago
• zetaops / websocket-tornado-rabbitmq-example

  View on GitHub
  Websocket based bidirectional communication between Tornado - RabbitMQ - worker processes
  ☆23Mar 4, 2016Updated 9 years ago
• joaopm33 / fundspy

  View on GitHub
  Download brazillian investment funds and their benchmarks data from CVM and analyze their performance with pre-built functions.
  ☆30Aug 13, 2022Updated 3 years ago
• dequinns / ScrapydArt

  View on GitHub
  在scrapyd基础上新增权限验证、爬虫运行信息统计、界面重构、，并增加排序、筛选过滤等多个API
  ☆112Nov 6, 2018Updated 7 years ago
• aaldaber / Distributed-Multi-User-Scrapy-System-with-a-Web-UI

  View on GitHub
  Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner
  ☆112May 11, 2018Updated 7 years ago
• jibudata / data-mover

  View on GitHub
  export snapshot to S3 and retore to workspace
  ☆11May 7, 2022Updated 3 years ago
• Lineblocs / internals-api

  View on GitHub
  internal API for call processing
  ☆11Jan 30, 2026Updated 2 weeks ago
• TeamHG-Memex / arachnado

  View on GitHub
  Web Crawling UI and HTTP API, based on Scrapy and Tornado
  ☆160Updated this week
• gnemoug / distribute_crawler

  View on GitHub
  使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
  ☆3,253Apr 18, 2017Updated 8 years ago
• aivarsk / scrapy-proxies

  View on GitHub
  Random proxy middleware for Scrapy
  ☆1,670Oct 1, 2019Updated 6 years ago
• ljanyst / scrapy-do

  View on GitHub
  A daemon for scheduling Scrapy spiders
  ☆66May 28, 2021Updated 4 years ago