DanielDaCosta / airflow-ml-prediction
Running ECS task for ML prediction orchestrated by Airflow
☆14Updated last year
Alternatives and similar repositories for airflow-ml-prediction:
Users that are interested in airflow-ml-prediction are comparing it to the libraries listed below
- Materials for the next course☆24Updated 2 years ago
- AWS Glue tutorial for data developers.☆23Updated 5 years ago
- ☆60Updated 3 years ago
- Full stack data engineering tools and infrastructure set-up☆51Updated 4 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆175Updated 3 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆60Updated last year
- Example repo to create end to end tests for data pipeline.☆23Updated 10 months ago
- ☆64Updated last week
- Bigdata on Kubernetes, Published by Packt☆30Updated 6 months ago
- 📒(GitBook) A curated list of awesome Data Engineering resources☆35Updated 3 weeks ago
- Docker environment that spins up MongoDB replica set, Spark, and Jupyter Lab. Example code uses PySpark and the MongoDB Spark Connector.☆40Updated 2 years ago
- Docker Airflow - Contains a docker compose file for Airflow 2.0☆65Updated 2 years ago
- ☆31Updated 8 months ago
- ☆16Updated 3 years ago
- Demo for GitHub Universe 2022☆12Updated 2 years ago
- ☆36Updated 2 years ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆56Updated last year
- Serverless ETL and Analytics with AWS Glue, published by Packt☆48Updated last year
- Build and deploy a serverless data pipeline on AWS with no effort.☆111Updated 2 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆87Updated 4 years ago
- Terraform module to setup Managed Workflows with Apache Airflow. (Airflow as managed service by AWS)☆35Updated 2 weeks ago
- Deploy of Airflow 2.0 using ECS Fargate and AWS CDK.☆14Updated 3 years ago
- Spark data pipeline that processes movie ratings data.☆28Updated 3 weeks ago
- Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR☆18Updated 8 months ago
- Essential PySpark for Scalable Data Analytics, published by Packt☆44Updated 2 years ago
- A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.☆38Updated 4 years ago
- A repository of sample code to show data quality checking best practices using Airflow.☆76Updated 2 years ago
- Course Material☆24Updated 2 years ago
- Building a Data Pipeline with an Open Source Stack☆53Updated 9 months ago
- ☆11Updated 5 months ago