fabiogouw / spark-aws-messagingLinks
A custom sink provider for Apache Spark that sends the content of a dataframe to an AWS SQS
☆23Updated last year
Alternatives and similar repositories for spark-aws-messaging
Users that are interested in spark-aws-messaging are comparing it to the libraries listed below
Sorting:
- Example code for running Spark and Hive jobs on EMR Serverless.☆168Updated 9 months ago
- ☆21Updated 3 years ago
- For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR☆67Updated 3 years ago
- Delta Lake examples☆230Updated last year
- ☆104Updated 9 months ago
- The AWS Glue Data Catalog is a fully managed, Apache Hive Metastore compatible, metadata repository. Customers can use the Data Catalog a…☆225Updated 7 months ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆181Updated 3 years ago
- Airflow Deployment on AWS ECS Fargate Using Cloudformation☆205Updated 3 years ago
- ☆269Updated last year
- Spark runtime on AWS Lambda☆111Updated 2 months ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆61Updated 2 years ago
- A Python Library to support running data quality rules while the spark job is running⚡☆190Updated this week
- This repository is for demonstrating the capability to do SQL-based UPDATES, DELETES, and INSERTS directly in the Data Lake using Amazon …☆18Updated 4 years ago
- 📚 Tech blogs & talks by companies that run Apache Flink in production☆181Updated 2 months ago
- A best practices guide for using AWS EMR. The guide will cover best practices on the topics of cost, performance, security, operational e…☆109Updated 3 weeks ago
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆44Updated 2 years ago
- Data Engineering com Apache Spark☆42Updated 4 years ago
- ☆80Updated 6 months ago
- Snowflake Data Source for Apache Spark.☆230Updated 2 weeks ago
- Delta Lake Documentation☆50Updated last year
- This repository contains the dbt-glue adapter☆135Updated last week
- ☆61Updated last year
- The Amazon S3 Tables catalog is a client library that bridges control plane operations provided by S3 Tables to engines like Apache Spark…☆139Updated 2 months ago
- BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.☆409Updated last week
- ☆18Updated last year
- A repository of sample code to accompany our blog post on Airflow and dbt.☆179Updated 2 years ago
- ☆23Updated 2 years ago
- Demo DAGs that show how to run dbt Core in Airflow using Cosmos☆64Updated 5 months ago
- Apache flink☆73Updated 3 months ago
- Enables synchronizing metadata changes (Create/Drop table/partition) from Hive Metastore to AWS Glue Data Catalog☆35Updated last year