canyousayyes / scrapy-web-crawler-by-rest-api
Example Scrapy project to crawl the web using the site's REST API
☆15Updated 5 years ago
Alternatives and similar repositories for scrapy-web-crawler-by-rest-api:
Users that are interested in scrapy-web-crawler-by-rest-api are comparing it to the libraries listed below
- boilerplate code to start with celery and rabbitmq in docker cluster☆19Updated 2 years ago
- Seeker - another job board aggregator.☆27Updated 4 years ago
- Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner☆113Updated 6 years ago
- ☆29Updated 3 years ago
- Pyppeteer integration for Scrapy☆59Updated 3 years ago
- Free & open source API service for obtaining information about +9600 universities worldwide.☆63Updated 3 years ago
- A scrapy pipeline which send items to Elastic Search server☆98Updated 7 years ago
- E-commerce Web Application written in Django with Payment Integration, Asyncronous task processing using Celery, Flower etc..☆11Updated 5 years ago
- More flexible and featured Frontera scheduler for Scrapy☆36Updated 2 months ago
- A Scrapy crawler for http://books.toscrape.com☆27Updated 7 years ago
- Asyncio web crawling framework. Work in progress.☆18Updated 5 months ago
- Highly scalable webcrawler for towardsdatascience.com by using Python, Selenium, Docker, Kubernetes and the infrastructure of the Google …☆25Updated 3 years ago
- Scrape websites asynchronously with Python 3.8+, Asyncio, & arsenic (aka Selenium for Async).☆56Updated 3 years ago
- Example frontera project☆12Updated 7 years ago
- Create a Serverless Web Application using Zappa, AWS Lambda, and Django.☆27Updated 2 years ago
- Zyte Automatic Extraction integration for Scrapy☆56Updated 2 years ago
- Software stack with latest Scrapy and updated deps☆63Updated 3 weeks ago
- ☆32Updated 5 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆51Updated 6 years ago
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 8 months ago
- Use the React CDN as well as Babel to make a Standalone React app without running `npx create-react-app`☆12Updated 5 years ago
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website☆42Updated 7 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆45Updated 3 years ago
- Flask based UI for displaying & segmenting a single database table☆15Updated 2 years ago
- Flask Boilerplate - Built with Automation Tools | AppSeed App Generator☆18Updated 3 years ago
- Code for the second edition of Django Design Patterns and Best Practices book by Arun Ravindran☆39Updated 2 years ago
- Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.☆21Updated 4 years ago
- Learn how to integrate a minimal FastAPI project with Airtable as our data store.☆26Updated 4 years ago
- Django seed project using postgres as primary database and elasticsearch as secondary db☆15Updated last year
- An example program that scrapes data from AllRecipes.com and store in Elasticsearch☆99Updated 6 years ago