Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner
☆112May 11, 2018Updated 7 years ago
Alternatives and similar repositories for Distributed-Multi-User-Scrapy-System-with-a-Web-UI
Users that are interested in Distributed-Multi-User-Scrapy-System-with-a-Web-UI are comparing it to the libraries listed below
Sorting:
- Simple Web UI for Scrapy spider management via Scrapyd☆50Jun 25, 2018Updated 7 years ago
- Digital store on top of Django Oscar.☆19Nov 30, 2024Updated last year
- a demo of scrapy + selenium☆21Sep 18, 2019Updated 6 years ago
- Creating Scrapy scrapers via the Django admin interface☆1,162Feb 19, 2022Updated 4 years ago
- admin ui for scrapy/open source scrapinghub☆2,778May 4, 2023Updated 2 years ago
- The Tweets2013 Internet Archive collection☆10Aug 7, 2020Updated 5 years ago
- Simple class for parsing left and right column for Yandex Wordstat via Direct API☆13Feb 27, 2022Updated 4 years ago
- ☆16Jan 27, 2021Updated 5 years ago
- Command line client for Scrapyd server☆778Dec 15, 2025Updated 2 months ago
- Respository for using session authentication with Django Rest Framework and React. Tutorial at https://medium.com/@kieron.mckenna/django-…☆19Oct 9, 2022Updated 3 years ago
- Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js☆3,500Oct 29, 2024Updated last year
- ☆20Aug 29, 2023Updated 2 years ago
- This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.☆1,231Nov 7, 2023Updated 2 years ago
- scrapy-ui☆16Feb 21, 2014Updated 12 years ago
- Scrapy, tianya, 天涯; scrapy django增量抓取天涯莲蓬鬼话全部帖子☆21Mar 20, 2025Updated 11 months ago
- MongoDB pipeline for Scrapy. This module supports both MongoDB in standalone setups and replica sets. scrapy-mongodb will insert the item…☆358Apr 6, 2021Updated 4 years ago
- HTTP API for Scrapy spiders☆881Feb 16, 2026Updated 2 weeks ago
- admin ui for scrapy/open source scrapinghub☆58Apr 28, 2021Updated 4 years ago
- 【爬虫】基于Scrapy开发的微博（评论、转发、点赞）爬虫，可以批量抓取。☆29Dec 1, 2016Updated 9 years ago
- A service daemon to run Scrapy spiders☆3,085Updated this week
- ☆29Apr 17, 2018Updated 7 years ago
- Python web visualize build on the awesome web framework sanic☆63Jul 12, 2017Updated 8 years ago
- Web Crawling UI and HTTP API, based on Scrapy and Tornado☆160Feb 10, 2026Updated 3 weeks ago
- Random proxy middleware for Scrapy☆1,671Oct 1, 2019Updated 6 years ago
- A Scrapy crawler for http://books.toscrape.com☆27May 26, 2017Updated 8 years ago
- Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls☆277Feb 26, 2025Updated last year
- A scalable frontier for web crawlers☆1,330Jun 6, 2025Updated 8 months ago
- Scrapy middleware which allows to crawl only new content☆79Feb 10, 2026Updated 3 weeks ago
- ☆19Nov 18, 2024Updated last year
- ☆10Sep 21, 2021Updated 4 years ago
- OSoMe API mashups☆11Jan 29, 2019Updated 7 years ago
- Rank Aggregation Algorithms☆12Jul 22, 2014Updated 11 years ago
- 基于Pymysql和Pyqt的图书管理系统 Book Management System based on Pymysql and Pyqt☆12Jan 13, 2023Updated 3 years ago
- European Parliament website Python scraper☆12Oct 19, 2016Updated 9 years ago
- RPC4Django is an XMLRPC and JSONRPC server for Django powered projects. Simply plug it into any existing Django project and you can make …☆55Oct 6, 2020Updated 5 years ago
- Source code for our AAAI 2020 paper P-SIF: Document Embeddings using Partition Averaging☆35May 2, 2020Updated 5 years ago
- PyQt5 based SQL query executor demo tutorial and reference. Uses DB-API and SQLite3. Performs syntax highlighting in QTextEdit using pygm…☆32Feb 21, 2021Updated 5 years ago
- Excel to Json online converter made with Python/Flask and React.js☆12Jan 4, 2023Updated 3 years ago
- ☆25Apr 10, 2019Updated 6 years ago