aws-samples / aws-glue-crawler-utilities
This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS CDK applications.
☆17Updated 2 years ago
Related projects: ⓘ
- Learn how to build an end-to-end streaming architecture to ingest, analyze, and visualize streaming data in near real-time☆34Updated 2 years ago
- Operational Data Processing Framework developed using AWS Glue and Apache Hudi. This framework is suitable for Data Lake and Modern Data …☆21Updated last year
- ☆26Updated 6 months ago
- Streaming ETL with Apache Flink and Amazon Kinesis Data Analytics☆64Updated 11 months ago
- A Pyspark job to handle upserts, conversion to parquet and create partitions on S3☆26Updated 4 years ago
- Collection of Cloud Formation Templates, Lambda Scripts and sample code required to provision an AWS Data Lake for a ReInvent Lab Exercis…☆26Updated 5 years ago
- A self-paced workshop designed to allow you to get hands on with building a real-time data platform using serverless technologies such as…☆22Updated 5 years ago
- Optimizing downstream data processing with Amazon Kinesis Data Firehose and Amazon EMR running Apache Spark☆13Updated last year
- This workshop will familiarize you with some of the key steps towards building an autonomous driving data lake and extracting images from…☆8Updated 2 years ago
- This repository contains ready-to-use notebook examples for a wide variety of use cases in Amazon EMR Studio.☆44Updated 10 months ago
- Kinesis Data Analytics Blueprints are a curated collection of Apache Flink applications. Each blueprint will walk you through how to solv…☆16Updated last year
- Design best practices for building scalable ETL (Extract-Transform-Load) and ELT (Extract-Load-Transform) data processing pipelines using…☆16Updated 4 years ago
- Enables synchronizing metadata changes (Create/Drop table/partition) from Hive Metastore to AWS Glue Data Catalog☆32Updated 9 months ago
- Replication utility for AWS Glue Data Catalog☆73Updated last month
- A serverless datalake project and framework based on AWS S3，Glue，Athena，MWAA and QuickSight. With a series of best practices, it guides y…☆16Updated last year
- As customers move from building data lakes and analytics on AWS to building machine learning solutions, one of their biggest challenges i…☆61Updated 5 years ago
- Data Profiler for AWS Glue Data Catalog application as described in the AWS Big Data Blog post "Build an automatic data profiling and rep…☆18Updated 4 years ago
- ☆16Updated 5 months ago
- The open source version of the Amazon EMR Management Guide. You can submit feedback & requests for changes by submitting issues in this r…☆61Updated last year
- This solution helps you deploy ETL jobs on data lake using CDK Pipelines.☆66Updated 2 years ago
- A Personalized 'Shop-by-Style' Experience via PyTorch on Amazon SageMaker and Amazon Neptune☆24Updated 3 years ago
- A project template for developing BYOD docker images for use in Amazon SageMaker.☆19Updated 4 years ago
- AWS Glue tutorial for data developers.☆23Updated 5 years ago
- A solution describing data-processing design pattern for streaming data through Kinesis and Spark Streaming at real-time.☆38Updated 3 months ago
- This project uses the open-source model Mixtral 8x7B Instruct, deployed in Amazon SageMaker or invoked via API on Amazon Bedrock, to enab…☆18Updated 2 months ago
- This GitHub project provides a series of lab exercises which help users get started using the Redshift platform.☆53Updated 3 years ago
- ☆46Updated 5 months ago
- Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3☆17Updated last week
- Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with AWS Glue Streaming and DMS☆23Updated last week
- Sample Apache Beam pipeline that can be deployed to Amazon Managed Service for Apache Flink. It reads taxi events from a Kinesis data str…☆46Updated 10 months ago