canyousayyes / scrapy-web-crawler-by-rest-api
Example Scrapy project to crawl the web using the site's REST API
☆15Updated 5 years ago
Related projects ⓘ
Alternatives and complementary repositories for scrapy-web-crawler-by-rest-api
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website☆42Updated 7 years ago
- More flexible and featured Frontera scheduler for Scrapy☆35Updated 10 months ago
- a Django waiting list app☆28Updated 4 years ago
- Generates the files needed for a production ready Django deployment in Docker. Custom user model, PostgreSQL database backend, uWSGI Pyth…☆23Updated 5 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆50Updated 6 years ago
- Flask Boilerplate - Built with Automation Tools | AppSeed App Generator☆18Updated 3 years ago
- a document management app for Django☆65Updated 3 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆44Updated 3 years ago
- Zyte Automatic Extraction integration for Scrapy☆55Updated 2 years ago
- Detect and classify pagination links☆14Updated 4 years ago
- Techcrunch Incremental Scrapy Spider With MongoDB☆16Updated 5 years ago
- Personal blog site using Wagtail CMS☆20Updated last year
- A simple python tool that generates a requests/bs4 based web scraper☆26Updated 2 years ago
- Run a Scrapy spider programmatically from a script or a Celery task - no project required.☆121Updated 5 months ago
- Extensions for using Scrapy on Amazon AWS☆32Updated 11 years ago
- A scrapy pipeline which send items to Elastic Search server☆98Updated 6 years ago
- ☆32Updated 7 years ago
- ☆49Updated 2 years ago
- Kubernetes on DigitalOcean☆12Updated 3 years ago
- A Docker implementation of Celery running on Flask, managed with supervisord.☆36Updated 6 years ago
- blog post☆19Updated 5 years ago
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 6 months ago
- Example frontera project☆12Updated 7 years ago
- Free & open source API service for obtaining information about +9600 universities worldwide.☆62Updated 3 years ago
- 📑⚙️ Python/Django reference implementation of the ERAV data model☆21Updated 5 years ago
- Running Flask on Docker Swarm☆37Updated 7 months ago
- Backend, modern REST API for obtaining match and odds data crawled from multiple sites. Using FastAPI, MongoDB as database, Motor as asyn…☆61Updated last year
- Software stack with latest Scrapy and updated deps☆62Updated this week
- User management model mixins and api views.☆58Updated last year