aws-samples / iceberg-streaming-examplesLinks
This repo contains examples of high throughput ingestion using Apache Spark and Apache Iceberg. These examples cover IoT and CDC scenarios using best practices. The code can be deployed into any Spark compatible engine like Amazon EMR Serverless or AWS Glue. A fully local developer environment is also provided.
☆26Updated last month
Alternatives and similar repositories for iceberg-streaming-examples
Users that are interested in iceberg-streaming-examples are comparing it to the libraries listed below
Sorting:
- Sample code to collect Apache Iceberg metrics for table monitoring☆28Updated last year
- This repository contains ready-to-use notebook examples for a wide variety of use cases in Amazon EMR Studio.☆52Updated last year
- Spark runtime on AWS Lambda☆109Updated last month
- Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR☆17Updated 4 months ago
- ☆11Updated 9 months ago
- Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3☆24Updated 11 months ago
- This repository contains the dbt-glue adapter☆131Updated last week
- ☆73Updated last year
- Example code for running Spark and Hive jobs on EMR Serverless.☆167Updated 7 months ago
- dbt (data build tool) projects targeting AWS analytics services (redshift, glue, emr, athena) and open table formats☆30Updated 2 years ago
- ☆54Updated last month
- Amazon Managed Workflows for Apache Airflow (MWAA) Examples repository contains example DAGs, requirements.txt, plugins, and CloudFormati…☆115Updated last month
- Unity Catalog UI☆42Updated 11 months ago
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- Source code for the post, 'Getting Started with Data Analysis on AWS, using S3, Glue, Amazon Athena, and QuickSight'☆28Updated 4 years ago
- Describes the concepts of lambda architecture and the actual deployment process with an example of building a serverless business intelli…☆15Updated 2 months ago
- ☆16Updated 2 years ago
- An open-source framework that simplifies implementation of data solutions.☆141Updated this week
- ☆22Updated 6 months ago
- Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work☆47Updated 3 years ago
- ☆16Updated last month
- Best practices and recommendations for getting started with Amazon EMR on EKS.☆66Updated 2 months ago
- ☆32Updated last year
- ☆58Updated this week
- This solution helps you deploy ETL jobs on data lake using CDK Pipelines.☆69Updated 3 years ago
- ☆18Updated last year
- Build DataOps platform with Apache Airflow and dbt on AWS☆59Updated 4 years ago
- Demonstrating the capabilities of DuckDB as a transformation engine for data lakes☆30Updated 10 months ago
- This repository demonstrates the construction of a state-of-the-art multimodal search engine, leveraging Amazon Titan Embeddings, Amazon …☆34Updated 3 months ago
- Operational Data Processing Framework developed using AWS Glue and Apache Hudi. This framework is suitable for Data Lake and Modern Data …☆23Updated last year