viktorfa / scrapy-fargate-sls-guide
A guide/tutorial for running Scrapy with a serverless paradigm
☆32Updated last year
Alternatives and similar repositories for scrapy-fargate-sls-guide:
Users that are interested in scrapy-fargate-sls-guide are comparing it to the libraries listed below
- Python clients for Zyte AutoExtract API☆40Updated 3 years ago
- Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.☆75Updated 2 years ago
- A job scraper using the Scrapy framework☆17Updated 7 years ago
- Scrapy Deployed on AWS Lambda☆11Updated 2 years ago
- Techniques for Scraping the Web in Python☆26Updated 6 years ago
- This repo demonstrates how to load a sample Parquet formatted file from an AWS S3 Bucket. A python job will then be submitted to a Apach…☆19Updated 8 years ago
- Zyte Automatic Extraction integration for Scrapy☆56Updated 3 years ago
- More flexible and featured Frontera scheduler for Scrapy☆36Updated 2 months ago
- sample code for tech blog post "Porting Flask to FastAPI for ML Model Serving"☆29Updated last year
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website☆42Updated 7 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆45Updated 3 years ago
- Basic tutorial of using Apache Airflow☆36Updated 6 years ago
- A Serverless Crawler For Real State Data in Vancouver Using AWS Lambda, Dynamo, RDS MySQL and CloudWatch☆80Updated 7 years ago
- Python starter project for Serverless Framework☆38Updated 3 years ago
- JavaScript support and proxy rotation for Scrapy with ScrapingBee.☆38Updated 9 months ago
- Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends☆55Updated last year
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆84Updated 5 years ago
- Easily interact with cloud (AWS) in your Data Science workflow.☆20Updated 2 years ago
- Example code that launches a docker container on AWS Fargate from AWS Lambda☆18Updated 7 years ago
- Using Luigi to create a Machine Learning Pipeline using the Rossman Sales data from Kaggle☆33Updated 8 years ago
- boilerplate code to start with celery and rabbitmq in docker cluster☆20Updated 2 years ago
- 💾 Script to import issues from a JIRA instance into a database.☆56Updated 2 years ago
- A demonstration of deploying flask on serverless (FaaS)☆12Updated 2 years ago
- Software stack with latest Scrapy and updated deps☆63Updated last week
- ☆14Updated 2 years ago
- Course materials and handouts for EVE: Building RESTful MongoDB-backed APIs course☆63Updated 6 years ago
- Event-driven, Serverless Architectures with AWS Lambda, SQS, DynamoDB, and API Gateway☆36Updated 3 years ago
- A self-paced workshop designed to allow you to get hands on with building a real-time data platform using serverless technologies such as…☆22Updated 6 years ago
- Web Scraping Craigslist's Engineering Jobs in NY with Scrapy☆66Updated 7 years ago
- The Summarlight Chrome Extension highlights the most important parts of posts/stories/articles.☆26Updated 5 years ago