adidas / m3d-engineLinks
M3D Engine is a Spark application for the development of scalable data transformations and ingestions in data lakes.
☆18Updated 4 years ago
Alternatives and similar repositories for m3d-engine
Users that are interested in m3d-engine are comparing it to the libraries listed below
Sorting:
- Metadata Driven Development (m3d) is a cloud and platform agnostic framework for the automated creation, management and governance of dat…☆33Updated 2 years ago
- Auto-generated Diagrams from Airflow DAGs. 🔮 🪄☆349Updated this week
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆96Updated last week
- FADI - Ingest, store and analyse big data flows☆46Updated last year
- Support for generating modern platforms dynamically with services such as Kafka, Spark, Streamsets, HDFS, ....☆76Updated last week
- ☆27Updated last year
- Egeria's Guidance on Governance as well as large media files such as presentations and movies☆106Updated 2 years ago
- Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.☆126Updated 4 years ago
- Terraform / NiFi on the Google Cloud Platform☆28Updated 10 months ago
- ☆97Updated 2 years ago
- Prometheus Exporter for Airflow☆161Updated last year
- The NiFiKop NiFi Kubernetes operator makes it easy to run Apache NiFi on Kubernetes. Apache NiFI is a free, open-source solution that sup…☆128Updated 3 years ago
- Curated list of resources about Apache Airflow☆19Updated 4 years ago
- Delta Lake Documentation☆50Updated last year
- Airflow declarative DAGs via YAML☆133Updated 2 years ago
- Repository of helm charts for deploying DataHub on a Kubernetes cluster☆194Updated this week
- Use SQL to build ELT pipelines on a data lakehouse.☆288Updated 3 years ago
- A convenient Python wrapper for Apache NiFi☆266Updated this week
- Airflow on Kubernetes Operator☆86Updated 2 years ago
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆100Updated 2 years ago
- Example of a single node Presto with Azure Data Lake Store (ADLS) and Azure Storage Blob (WASB) access via Hive metastore☆19Updated 5 years ago
- Cloudera Director sample code☆61Updated 5 years ago
- Grafana dashboards and StatsD exporter config for Airflow monitoring☆287Updated last year
- Spark on Kubernetes infrastructure Helm charts repo☆204Updated 2 years ago
- A Helm chart to install Apache Airflow on Kubernetes☆289Updated this week
- Spark on Kubernetes samples☆20Updated 4 years ago
- Sample Airflow DAGs☆63Updated 2 years ago
- ☆11Updated 6 years ago
- Setup for running Trino with Hive Metastore on Kubernetes☆103Updated 3 years ago
- 🚀 Deploy Kubeflow on AWS EKS with Terraform 🤖☆65Updated 2 years ago