viktorfa / scrapy-fargate-sls-guideLinks
A guide/tutorial for running Scrapy with a serverless paradigm
☆33Updated 2 years ago
Alternatives and similar repositories for scrapy-fargate-sls-guide
Users that are interested in scrapy-fargate-sls-guide are comparing it to the libraries listed below
Sorting:
- Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.☆75Updated 3 years ago
- A job scraper using the Scrapy framework☆16Updated 7 years ago
- Scraping tweets quickly using celery, RabbitMQ and Docker cluster☆48Updated 2 years ago
- A solution enabling customers to quickly deploy an architecture to identify and mask sensitive health data☆26Updated 2 years ago
- A demonstration of deploying flask on serverless (FaaS)☆12Updated 2 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆44Updated 4 years ago
- Python starter project for Serverless Framework☆38Updated 4 years ago
- Python Lambda Chrome Automation (naming pending)☆295Updated 9 months ago
- Basic tutorial of using Apache Airflow☆36Updated 6 years ago
- Serverless Template for a Flask application☆80Updated 6 years ago
- A Serverless Crawler For Real State Data in Vancouver Using AWS Lambda, Dynamo, RDS MySQL and CloudWatch☆80Updated 7 years ago
- A small Python module containing quick utility functions for standard ETL processes.☆36Updated last week
- ☆16Updated 3 years ago
- Scrapy project boilerplate done right☆48Updated 5 months ago
- Using Amazon Comprehend, Amazon Elasticsearch with Kibana, Amazon S3, Amazon Cognito to search over large number of documents.☆24Updated last year
- A project providing a Lambda Layer that provides SQLite support in Python3.6 Lambdas☆25Updated 6 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆51Updated 7 years ago
- Example code that launches a docker container on AWS Fargate from AWS Lambda☆18Updated 7 years ago
- More flexible and featured Frontera scheduler for Scrapy☆37Updated last month
- A Pythonic API for Amazon's States Language for defining AWS Step Functions☆8Updated 2 years ago
- 🚨 Simple, self-contained fraud detection system built with Apache Kafka and Python☆88Updated 6 years ago
- The Summarlight Chrome Extension highlights the most important parts of posts/stories/articles.☆26Updated 6 years ago
- Small demo for a "search-as-you-type" app in AngularJS + Python/Flask + Elasticsearch☆69Updated 7 years ago
- Python clients for Zyte AutoExtract API☆40Updated 3 years ago
- An example of a web scraper running on AWS Lambda and Lambda Layers☆137Updated 3 years ago
- Serverless Framework Plugin to package python functions and their requirements☆80Updated 2 years ago
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website☆44Updated 7 years ago
- Build and Deploy A Serverless Data Pipeline on AWS☆26Updated 2 years ago
- This repo demonstrates how to load a sample Parquet formatted file from an AWS S3 Bucket. A python job will then be submitted to a Apach…☆19Updated 9 years ago
- Apify actor to run web spiders written in Python in the Scrapy library☆11Updated 2 years ago