provectus / streaming-data-platformLinks
☆24Updated 3 years ago
Alternatives and similar repositories for streaming-data-platform
Users that are interested in streaming-data-platform are comparing it to the libraries listed below
Sorting:
- Reference Dockerfiles for production usage☆24Updated 5 years ago
- For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR☆67Updated 3 years ago
- Swiss Army Kube (SAK) is an open-source IaC (Infrastructure as Code) collection of services for quick, easy, and controllable deployment …☆148Updated 3 weeks ago
- Airflow declarative DAGs via YAML☆133Updated 2 years ago
- Data Quality Gate based on AWS☆57Updated last year
- Common Transforms for Kafka Connect.☆169Updated 5 months ago
- Deploy Presto on the cloud easily, using Terraform and Packer☆45Updated 2 years ago
- Rokku project. This project acts as a proxy on top of any S3 storage solution providing services like authentication, authorization, shor…☆70Updated 3 months ago
- Snowflake Kafka Connector (Sink Connector)☆162Updated this week
- Kafka Connect Store Partitioner by custom fields and time☆42Updated 4 years ago
- Aiven's collection of Single Message Transformations (SMTs) for Apache Kafka Connect☆87Updated 2 months ago
- Reference architecture for real-time stream processing with Apache Flink on Amazon EMR, Amazon Kinesis, and Amazon Elasticsearch Service.☆70Updated last year
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆97Updated last week
- A best practices guide for using AWS EMR. The guide will cover best practices on the topics of cost, performance, security, operational e…☆109Updated 2 months ago
- ☆98Updated 2 years ago
- Enables synchronizing metadata changes (Create/Drop table/partition) from Hive Metastore to AWS Glue Data Catalog☆35Updated 2 years ago
- An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR☆175Updated 6 months ago
- Superglue is a lineage-tracking tool built to help visualize the propagation of data through complex pipelines composed of tables, jobs …☆159Updated 3 years ago
- Data validation library for PySpark 3.0.0☆33Updated 3 years ago
- Testing LLMs and RAG configurations at scale using an OpenAI Reflector☆11Updated 11 months ago
- ☆63Updated last year
- A VS Code Extension to make it easier to manage and develop Spark jobs on EMR☆39Updated 9 months ago
- AWS Glue Schema Registry Client library provides serializers / de-serializers for applications to integrate with AWS Glue Schema Registry…☆147Updated last month
- Schema Registry☆17Updated last year
- Spark ETL example processing New York taxi rides public dataset on EKS☆44Updated 2 years ago
- JSON schema parser for Apache Spark☆82Updated 3 years ago
- An implementation of the DatasourceV2 interface of Apache Spark™ for writing Spark Datasets to Apache Druid™.☆43Updated 2 months ago
- ☆53Updated 6 years ago
- Experiments and demonstrations of AVRO, Protobuf serialisation☆61Updated 3 years ago
- Streaming ETL with Apache Flink and Amazon Kinesis Data Analytics☆65Updated 2 years ago