Run a Scrapy spider programmatically from a script or a Celery task - no project required.
☆121Jun 4, 2024Updated last year
Alternatives and similar repositories for scrapyscript
Users that are interested in scrapyscript are comparing it to the libraries listed below
Sorting:
- A daemon for scheduling Scrapy spiders☆66May 28, 2021Updated 4 years ago
- Scrapy project boilerplate done right☆48Updated this week
- Common interface for data container classes☆68Feb 23, 2026Updated last week
- This web crawler can be customized to scrape almost all types of websites.☆11Dec 31, 2021Updated 4 years ago
- Code and data for the research team scraping charter websites using scrapy, requests, Selenium, and wget with Python, shell, and Docker. …☆12Feb 8, 2023Updated 3 years ago
- HTTP API for Scrapy spiders☆881Feb 16, 2026Updated 2 weeks ago
- ☆13Apr 9, 2025Updated 10 months ago
- https://mimesniff.spec.whatwg.org/ implementation for Python☆13Jan 16, 2024Updated 2 years ago
- A service daemon to run Scrapy spiders☆3,085Updated this week
- SearchGar - An actual Search Engine made using Python☆18Sep 1, 2022Updated 3 years ago
- Scrapy Extension for monitoring spiders execution.☆553Updated this week
- 📑⚙️ Python/Django reference implementation of the ERAV data model☆21May 18, 2019Updated 6 years ago
- A Scrapy extension to log items coverage when the spider shuts down☆19Apr 11, 2020Updated 5 years ago
- WebThings framework.☆20Nov 25, 2022Updated 3 years ago
- Library designed to replace the SQLite backend by a MongoDB backend on Scrapy queue management☆17Sep 2, 2017Updated 8 years ago
- Page Object pattern for Scrapy☆127Jan 28, 2026Updated last month
- scraping and querying documents for LLMs☆24Oct 6, 2025Updated 4 months ago
- Template starter for Django + Inertia.js + Svelte.☆20Jun 5, 2023Updated 2 years ago
- Maintenance/updates now handled by brillgen/django-config-gen☆27Jun 28, 2018Updated 7 years ago
- ☆13Nov 14, 2023Updated 2 years ago
- Canadian legislative scrapers☆35Feb 13, 2026Updated 2 weeks ago
- A scrapy extension to sync `.scrapy` folder to an S3 bucket☆18Mar 28, 2022Updated 3 years ago
- Crochet-based blocking API for Scrapy.☆46Feb 24, 2017Updated 9 years ago
- This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.☆1,231Nov 7, 2023Updated 2 years ago
- Framework for rapid building CRUD based applications | Django, Python☆21Apr 20, 2019Updated 6 years ago
- ☆22Jan 11, 2023Updated 3 years ago
- A cookiecutter template for a Sphinx extension☆28Jan 11, 2024Updated 2 years ago
- A scrapy extension to store requests and responses information in storage service☆27Mar 11, 2022Updated 3 years ago
- Command line client for Scrapyd server☆778Dec 15, 2025Updated 2 months ago
- admin ui for scrapy/open source scrapinghub☆58Apr 28, 2021Updated 4 years ago
- A scalable frontier for web crawlers☆1,330Jun 6, 2025Updated 8 months ago
- Scrapy middleware to handle javascript pages using selenium☆958Jul 8, 2024Updated last year
- File download and upload support for Django REST framework☆27Dec 26, 2022Updated 3 years ago
- Django wrapper for intercooler.js - AJAX With Attributes: There is no need to be complex.☆31Nov 13, 2019Updated 6 years ago
- Combine XPath, CSS Selectors and JSONPath for Web data extracting.☆29Dec 25, 2024Updated last year
- admin ui for scrapy/open source scrapinghub☆2,778May 4, 2023Updated 2 years ago
- A guide/tutorial for running Scrapy with a serverless paradigm☆33Mar 4, 2023Updated 3 years ago
- FedEasy is an intuitive powerful yet simple to use Federated Learning framework. Our goal is to streamline the process of setting up and …☆13Feb 25, 2026Updated last week
- Python 3.6+ Asyncio PostgreSQL query builder and model☆67Feb 19, 2023Updated 3 years ago