adidas / m3d-engineLinks
M3D Engine is a Spark application for the development of scalable data transformations and ingestions in data lakes.
☆19Updated 4 years ago
Alternatives and similar repositories for m3d-engine
Users that are interested in m3d-engine are comparing it to the libraries listed below
Sorting:
- Metadata Driven Development (m3d) is a cloud and platform agnostic framework for the automated creation, management and governance of dat…☆33Updated 2 years ago
- ☆27Updated 2 years ago
- Accompanying code examples for webinar and blog post "three ways to run airflow on kubernetes"☆15Updated 5 years ago
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆97Updated last week
- Auto-generated Diagrams from Airflow DAGs. 🔮 🪄☆355Updated 2 weeks ago
- Data Profiler for AWS Glue Data Catalog application as described in the AWS Big Data Blog post "Build an automatic data profiling and rep…☆20Updated 5 years ago
- Egeria's Guidance on Governance as well as large media files such as presentations and movies☆107Updated 3 years ago
- Terraform plans & commands to provision Azure VMSS and VM from a VM image on demand or from a Jenkins pipeline.☆27Updated 7 years ago
- Dremio Container Tools☆164Updated 5 months ago
- Spark on Kubernetes samples☆20Updated 4 years ago
- Sample Airflow DAGs☆65Updated 3 years ago
- Aiven's S3 Sink Connector for Apache Kafka®☆71Updated last year
- Performance optimization for Spark running on Kubernetes☆88Updated 5 years ago
- A Table format agnostic data sharing framework☆42Updated 2 years ago
- Delta Lake Documentation☆53Updated last year
- The Internals of Spark on Kubernetes☆72Updated 3 years ago
- spark on kubernetes☆104Updated 2 years ago
- Support for generating modern platforms dynamically with services such as Kafka, Spark, Streamsets, HDFS, ....☆80Updated last week
- Use SQL to build ELT pipelines on a data lakehouse.☆288Updated 3 years ago
- Databricks Migration Tools☆43Updated 4 years ago
- Spark ETL example processing New York taxi rides public dataset on EKS☆44Updated 3 years ago
- Azure Deployments using Terraform☆30Updated 3 years ago
- Road to Azure Data Engineer Part-II: DP-201 - Designing an Azure Data Solution☆19Updated 5 years ago
- A simple Spark-powered ETL framework that just works 🍺☆183Updated 4 months ago
- Yet Another (Spark) ETL Framework☆21Updated 2 years ago
- Ambari stack service for installing and managing Apache Airflow on HDP cluster☆58Updated 7 years ago
- FADI - Ingest, store and analyse big data flows☆46Updated last year
- A repository to store recipes, custom sources, transformations and other things to make your DataHub experience magical☆12Updated 3 years ago
- Delta Lake examples☆238Updated last year
- ☆100Updated 2 years ago