DanielDaCosta / airflow-ml-prediction
Running ECS task for ML prediction orchestrated by Airflow
☆14Updated last year
Related projects ⓘ
Alternatives and complementary repositories for airflow-ml-prediction
- ☆60Updated 2 years ago
- Spark data pipeline that processes movie ratings data.☆27Updated 3 weeks ago
- Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work☆47Updated 2 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆170Updated 2 years ago
- AWS Glue tutorial for data developers.☆23Updated 5 years ago
- This repository is for demonstrating the capability to do SQL-based UPDATES, DELETES, and INSERTS directly in the Data Lake using Amazon …☆16Updated 3 years ago
- Terraform module to setup Managed Workflows with Apache Airflow. (Airflow as managed service by AWS)☆33Updated 3 weeks ago
- Materials for the next course☆22Updated last year
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆57Updated last year
- ☆53Updated 11 months ago
- Example code for running Spark and Hive jobs on EMR Serverless.☆151Updated 2 weeks ago
- ☆31Updated last year
- A VS Code Extension to make it easier to manage and develop Spark jobs on EMR☆29Updated 2 months ago
- ☆43Updated 2 years ago
- Bare minimal Airflow on Kubernetes (Local, EKS, AKS)☆52Updated 4 years ago
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆44Updated last year
- Airflow helm chart for AWS EKS☆18Updated 3 years ago
- Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR☆17Updated 3 months ago
- Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation,…☆89Updated 2 years ago
- Demo for GitHub Universe 2022☆12Updated last year
- Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'☆119Updated last year
- Docker with Airflow and Spark standalone cluster☆244Updated last year
- A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.☆39Updated 3 years ago
- A terraform module that creates an airflow instance in AWS ECS.☆59Updated last year
- Delta-Lake, ETL, Spark, Airflow☆44Updated 2 years ago
- how to unit test your PySpark code☆28Updated 3 years ago
- This repository contains the dbt-glue adapter☆99Updated last week
- ☆85Updated last year
- Sample Airflow DAGs☆61Updated last year
- ☆44Updated last week