aws-samples / iceberg-streaming-examples
This repo contains examples of high throughput ingestion using Apache Spark and Apache Iceberg. These examples cover IoT and CDC scenarios using best practices. The code can be deployed into any Spark compatible engine like Amazon EMR Serverless or AWS Glue. A fully local developer environment is also provided.
☆24Updated 5 months ago
Alternatives and similar repositories for iceberg-streaming-examples:
Users that are interested in iceberg-streaming-examples are comparing it to the libraries listed below
- Sample code to collect Apache Iceberg metrics for table monitoring☆26Updated 8 months ago
- ☆11Updated 5 months ago
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3☆23Updated 7 months ago
- ☆16Updated 2 years ago
- This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS …☆19Updated 3 years ago
- Demo for GitHub Universe 2022☆12Updated 2 years ago
- Operational Data Processing Framework developed using AWS Glue and Apache Hudi. This framework is suitable for Data Lake and Modern Data …☆22Updated last year
- dbt (data build tool) projects targeting AWS analytics services (redshift, glue, emr, athena) and open table formats☆29Updated 2 years ago
- Unity Catalog UI☆40Updated 8 months ago
- aws-solutions-library-samples / guidance-for-using-google-tag-manager-for-server-side-website-analytics-on-awsThis Guidance helps you implement server-side tagging to collect event data and perform data analysis in near real-time.☆10Updated 2 months ago
- Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR☆18Updated last week
- This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (M…☆12Updated 7 months ago
- This repository contains ready-to-use notebook examples for a wide variety of use cases in Amazon EMR Studio.☆50Updated last year
- ☆16Updated last year
- ☆31Updated last year
- Yet Another (Spark) ETL Framework☆21Updated last year
- This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you…☆11Updated this week
- ☆19Updated last week
- aws-solutions-library-samples / guidance-for-preparing-and-validating-records-for-entity-resolution-on-awsThis Guidance demonstrates how to prepare and validate Personally Identifiable Information (PII) data, including physical address, phone,…☆9Updated 6 months ago
- JumpSpark - A modern cookiecutter template for pyspark projects with batteries included.☆10Updated last year
- Delta reader for the Ray open-source toolkit for building ML applications☆46Updated last year
- ☆30Updated last year
- Spark runtime on AWS Lambda☆107Updated 7 months ago
- ☆34Updated 2 years ago
- This repository contains the dbt-glue adapter☆117Updated this week
- Python implementation of Age-Partitioned Bloom Filter with S3 periodic backup support.☆11Updated 3 months ago
- AWS Quick Start Team☆18Updated 7 months ago
- This Guidance demonstrates how you can extend the data governance capabilities of Amazon DataZone to other Java Database Connectivity (JD…☆13Updated 6 months ago
- duckdb-etl-framework☆10Updated 4 months ago