adidas / m3d-engineLinks
M3D Engine is a Spark application for the development of scalable data transformations and ingestions in data lakes.
☆18Updated 4 years ago
Alternatives and similar repositories for m3d-engine
Users that are interested in m3d-engine are comparing it to the libraries listed below
Sorting:
- Egeria's Guidance on Governance as well as large media files such as presentations and movies☆106Updated 2 years ago
- Metadata Driven Development (m3d) is a cloud and platform agnostic framework for the automated creation, management and governance of dat…☆32Updated 2 years ago
- Auto-generated Diagrams from Airflow DAGs. 🔮 🪄☆348Updated this week
- Dremio Container Tools☆162Updated 3 weeks ago
- Support for generating modern platforms dynamically with services such as Kafka, Spark, Streamsets, HDFS, ....☆76Updated last week
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆96Updated this week
- ☆97Updated 2 years ago
- ☆27Updated last year
- Data Profiler for AWS Glue Data Catalog application as described in the AWS Big Data Blog post "Build an automatic data profiling and rep…☆20Updated 5 years ago
- Curated list of resources about Apache Airflow☆19Updated 4 years ago
- Superglue is a lineage-tracking tool built to help visualize the propagation of data through complex pipelines composed of tables, jobs …☆159Updated 2 years ago
- Unity Catalog UI☆42Updated last year
- Repository of helm charts for deploying DataHub on a Kubernetes cluster☆194Updated last week
- EverythingApacheNiFi☆115Updated last year
- Sample Airflow DAGs☆63Updated 2 years ago
- Prometheus Exporter for Airflow☆161Updated last year
- Airflow on Kubernetes Operator☆86Updated 2 years ago
- Grafana dashboards and StatsD exporter config for Airflow monitoring☆286Updated last year
- Fybrik☆132Updated last week
- A Helm chart to install Apache Airflow on Kubernetes☆286Updated this week
- The NiFiKop NiFi Kubernetes operator makes it easy to run Apache NiFi on Kubernetes. Apache NiFI is a free, open-source solution that sup…☆128Updated 3 years ago
- A simple Spark-powered ETL framework that just works 🍺☆182Updated this week
- Pylint plugin for static code analysis on Airflow code☆95Updated 4 years ago
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆100Updated 2 years ago
- Airflow plugin to export dag and task based metrics to Prometheus.☆260Updated 3 months ago
- Setup for running Trino with Hive Metastore on Kubernetes☆103Updated 3 years ago
- Databricks Migration Tools☆43Updated 4 years ago
- 🚀 Deploy Kubeflow on AWS EKS with Terraform 🤖☆65Updated 2 years ago
- 📆 Run, schedule, and manage your dbt jobs using Kubernetes.☆25Updated 7 years ago
- Terraform / NiFi on the Google Cloud Platform☆28Updated 10 months ago