fabiogouw / spark-aws-messaging
A custom sink provider for Apache Spark that sends the content of a dataframe to an AWS SQS
☆23Updated 9 months ago
Alternatives and similar repositories for spark-aws-messaging:
Users that are interested in spark-aws-messaging are comparing it to the libraries listed below
- Example code for running Spark and Hive jobs on EMR Serverless.☆161Updated 2 months ago
- ☆21Updated 3 years ago
- Data Engineering com Apache Spark☆42Updated 3 years ago
- Apache flink☆64Updated this week
- Spark runtime on AWS Lambda☆106Updated 6 months ago
- Materials for the next course☆24Updated 2 years ago
- Sample code to collect Apache Iceberg metrics for table monitoring☆25Updated 7 months ago
- This repository is for demonstrating the capability to do SQL-based UPDATES, DELETES, and INSERTS directly in the Data Lake using Amazon …☆16Updated 3 years ago
- One bite-sized tip or trick for Apache Flink practitioners every day leading up to Christmas Eve 2024.☆24Updated 3 months ago
- The AWS Glue Data Catalog is a fully managed, Apache Hive Metastore compatible, metadata repository. Customers can use the Data Catalog a…☆212Updated last week
- For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR☆66Updated 3 years ago
- Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes☆63Updated 2 years ago
- Automated data quality suggestions and analysis with Deequ on AWS Glue☆84Updated 2 years ago
- Football scouts from Cartola FC at a data lake with data warehouse and dashboard☆17Updated 3 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆172Updated 3 years ago
- Delta Lake examples☆221Updated 5 months ago
- Deploy of Airflow 2.0 using ECS Fargate and AWS CDK.☆14Updated 3 years ago
- 🐋 Docker image for AWS Glue Spark/Python☆23Updated last year
- A best practices guide for using AWS EMR. The guide will cover best practices on the topics of cost, performance, security, operational e…☆104Updated last week
- This repo provides the Kubernetes Helm chart for deploying Pyspark Notebook.☆17Updated 2 years ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆53Updated last year
- ☆71Updated 2 months ago
- This repository contains the dbt-glue adapter☆112Updated last week
- A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs☆41Updated 10 months ago
- ☆61Updated last year
- Performant Redshift data source for Apache Spark☆138Updated 2 months ago
- code-snippets☆10Updated this week
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆73Updated 3 years ago
- Code snippets for Data Engineering Design Patterns book☆75Updated last week
- A repository of sample code to show data quality checking best practices using Airflow.☆74Updated 2 years ago