aws-samples / iceberg-streaming-examples
This repo contains examples of high throughput ingestion using Apache Spark and Apache Iceberg. These examples cover IoT and CDC scenarios using best practices. The code can be deployed into any Spark compatible engine like Amazon EMR Serverless or AWS Glue. A fully local developer environment is also provided.
☆22Updated 4 months ago
Alternatives and similar repositories for iceberg-streaming-examples:
Users that are interested in iceberg-streaming-examples are comparing it to the libraries listed below
- Sample code to collect Apache Iceberg metrics for table monitoring☆25Updated 7 months ago
- ☆16Updated 2 years ago
- Operational Data Processing Framework developed using AWS Glue and Apache Hudi. This framework is suitable for Data Lake and Modern Data …☆21Updated last year
- Unity Catalog UI☆40Updated 6 months ago
- Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3☆23Updated 6 months ago
- Demo for GitHub Universe 2022☆12Updated 2 years ago
- dbt (data build tool) projects targeting AWS analytics services (redshift, glue, emr, athena) and open table formats☆29Updated last year
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- In this repository, we show how to get started with data lineage on AWS using OpenLineage. This is an AWS Cloud Development Kit project (…☆12Updated 7 months ago
- This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS …☆19Updated 3 years ago
- ☆31Updated last year
- A VS Code Extension to make it easier to manage and develop Spark jobs on EMR☆31Updated last month
- Demonstrating the capabilities of DuckDB as a transformation engine for data lakes☆22Updated 5 months ago
- DataHub on AWS demonstration resources☆10Updated 2 years ago
- Delta Lake Documentation☆49Updated 9 months ago
- Describes the concepts of lambda architecture and the actual deployment process with an example of building a serverless business intelli…☆13Updated last month
- Data Profiler for AWS Glue Data Catalog application as described in the AWS Big Data Blog post "Build an automatic data profiling and rep…☆19Updated 4 years ago
- This repository contains ready-to-use notebook examples for a wide variety of use cases in Amazon EMR Studio.☆50Updated last year
- This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you…☆11Updated this week
- Example project using DBT, Databricks and AdventureWorks sample database☆11Updated 2 years ago
- Using TypeScript and the AWS CDK, you can integrate Knowledge Bases into Amazon Bedrock to provide foundation models with contextual data…☆12Updated 10 months ago
- ☆13Updated 7 months ago
- This Guidance demonstrates how you can extend the data governance capabilities of Amazon DataZone to other Java Database Connectivity (JD…☆13Updated 5 months ago
- AWS Glue Configurable Test Data Generator for S3 Data Lakes and DynamoDB☆16Updated last year
- ☆19Updated 5 months ago