aws-samples / iceberg-streaming-examplesLinks
This repo contains examples of high throughput ingestion using Apache Spark and Apache Iceberg. These examples cover IoT and CDC scenarios using best practices. The code can be deployed into any Spark compatible engine like Amazon EMR Serverless or AWS Glue. A fully local developer environment is also provided.
☆25Updated 6 months ago
Alternatives and similar repositories for iceberg-streaming-examples
Users that are interested in iceberg-streaming-examples are comparing it to the libraries listed below
Sorting:
- Sample code to collect Apache Iceberg metrics for table monitoring☆27Updated 9 months ago
- ☆11Updated 6 months ago
- ☆15Updated 2 years ago
- Unity Catalog UI☆40Updated 8 months ago
- Demo for GitHub Universe 2022☆12Updated 2 years ago
- Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3☆23Updated 8 months ago
- ☆19Updated last month
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- ☆32Updated last year
- Operational Data Processing Framework developed using AWS Glue and Apache Hudi. This framework is suitable for Data Lake and Modern Data …☆22Updated last year
- In this repository, we show how to get started with data lineage on AWS using OpenLineage. This is an AWS Cloud Development Kit project (…☆12Updated 10 months ago
- aws-solutions-library-samples / guidance-for-preparing-and-validating-records-for-entity-resolution-on-awsThis Guidance demonstrates how to prepare and validate Personally Identifiable Information (PII) data, including physical address, phone,…☆9Updated 7 months ago
- This solution helps you deploy ETL processes and data storage resources to create an Insurance Lake using Amazon S3 buckets for storage, …☆26Updated last month
- This repository contains ready-to-use notebook examples for a wide variety of use cases in Amazon EMR Studio.☆51Updated last year
- Demonstrating the capabilities of DuckDB as a transformation engine for data lakes☆27Updated 7 months ago
- ☆22Updated 3 months ago
- ☆46Updated this week
- ☆10Updated 8 months ago
- This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you…☆11Updated this week
- ☆18Updated 11 months ago
- dbt (data build tool) projects targeting AWS analytics services (redshift, glue, emr, athena) and open table formats☆29Updated 2 years ago
- Spark runtime on AWS Lambda☆107Updated 8 months ago
- Delta Lake Documentation☆49Updated 11 months ago
- ☆27Updated 2 months ago
- AWS Quick Start Team☆19Updated 8 months ago
- Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR☆18Updated last month
- Python implementation of Age-Partitioned Bloom Filter with S3 periodic backup support.☆11Updated 4 months ago
- A CLI to manage and monitor permissions in AWS Lake Formation☆26Updated 2 years ago
- This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS …☆19Updated 3 years ago
- Demo code to illustrate the execution of PyTest unit test cases for AWS Glue jobs in AWS CodePipeline using AWS CodeBuild projects☆46Updated 6 months ago