dacort / modern-data-lake-storage-layers
Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work
☆47Updated 2 years ago
Related projects: ⓘ
- Example code for running Spark and Hive jobs on EMR Serverless.☆149Updated last month
- Spark runtime on AWS Lambda☆92Updated last month
- Automated data quality suggestions and analysis with Deequ on AWS Glue☆83Updated last year
- This repository contains de dbt-glue adapter☆95Updated this week
- This repository contains ready-to-use notebook examples for a wide variety of use cases in Amazon EMR Studio.☆44Updated 10 months ago
- Amazon Managed Workflows for Apache Airflow (MWAA) Examples repository contains example DAGs, requirements.txt, plugins, and CloudFormati…☆106Updated this week
- Spark ETL example processing New York taxi rides public dataset on EKS☆42Updated last year
- PySpark data-pipeline testing and CICD☆28Updated 3 years ago
- Project files for the post: Running PySpark Applications on Amazon EMR: Methods for Interacting with PySpark on Amazon Elastic MapReduce.☆38Updated 2 years ago
- ☆17Updated 3 months ago
- This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concu…☆76Updated 5 years ago
- Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for A…☆41Updated 2 years ago
- ☆34Updated last year
- A best practices guide for using AWS EMR. The guide will cover best practices on the topics of cost, performance, security, operational e…☆98Updated 3 weeks ago
- Build DataOps platform with Apache Airflow and dbt on AWS☆51Updated 3 years ago
- For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR☆65Updated 2 years ago
- Demo for GitHub Universe 2022☆12Updated last year
- Replication utility for AWS Glue Data Catalog☆73Updated last month
- ☆85Updated 10 months ago
- Streaming ETL with Apache Flink and Amazon Kinesis Data Analytics☆64Updated 11 months ago
- The open source version of the Amazon EMR Management Guide. You can submit feedback & requests for changes by submitting issues in this r…☆61Updated last year
- Delta Lake Documentation☆45Updated 3 months ago
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆43Updated last year
- This repository is for demonstrating the capability to do SQL-based UPDATES, DELETES, and INSERTS directly in the Data Lake using Amazon …☆16Updated 3 years ago
- ☆36Updated 2 months ago
- In this workshop you will launch an Amazon Redshift cluster in your AWS account and load sample data ~ 100GB using TPCH dataset. You will…☆24Updated 5 years ago
- ☆19Updated this week
- Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR☆17Updated last month
- Demo code to illustrate the execution of PyTest unit test cases for AWS Glue jobs in AWS CodePipeline using AWS CodeBuild projects☆38Updated 3 months ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆166Updated 2 years ago