argoproj / data-pipelineLinks
☆36Updated 8 years ago
Alternatives and similar repositories for data-pipeline
Users that are interested in data-pipeline are comparing it to the libraries listed below
Sorting:
- An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR☆176Updated 8 months ago
- Rokku project. This project acts as a proxy on top of any S3 storage solution providing services like authentication, authorization, shor…☆70Updated 5 months ago
- Docker image to submit Spark applications☆38Updated 8 years ago
- Setup for running Trino with Hive Metastore on Kubernetes☆103Updated 3 years ago
- spark on kubernetes☆104Updated 2 years ago
- ☆128Updated 5 years ago
- Kubernetes custom controller and CRDs to managing Airflow☆298Updated 5 years ago
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆97Updated this week
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆103Updated 3 years ago
- Multiple node presto cluster on docker container☆126Updated 3 years ago
- Data ingestion library for Amundsen to build graph and search index☆204Updated last year
- Fast iterative local development and testing of Apache Airflow workflows☆204Updated last month
- One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)☆120Updated 4 years ago
- Front-end service library for Amundsen☆278Updated this week
- A guide to running Airflow on Kubernetes☆174Updated 6 years ago
- Airflow Unit Tests and Integration Tests☆261Updated 3 years ago
- Spark on Kubernetes infrastructure Helm charts repo☆203Updated 3 years ago
- A K8s-based infrastructure for analytics☆24Updated 6 years ago
- Airflow configuration for Telemetry☆199Updated this week
- Dockerized Hadoop/Minio/Hive/Presto stack☆37Updated last year
- SQL data model for working with Snowplow web data. Supports Redshift and Looker. Snowflake and BigQuery coming soon☆60Updated 5 years ago
- A plugin to Apache Airflow to allow you to run Spark Submit Commands as an Operator☆73Updated 6 years ago
- ☆113Updated last year
- Builds Airflow DAGs from configuration files. Powers all DAGs on the Etsy Data Platform☆259Updated 2 years ago
- Airflow workflow management platform chef cookbook.☆70Updated 6 years ago
- A docker image and kubernetes config files to run Airflow on Kubernetes☆655Updated 6 years ago
- An Integrated and collaborative cloud environment for building and running Spark applications on PKS/Kubernetes☆84Updated 5 years ago
- DataQuality for BigData☆147Updated 2 years ago
- Tool to automate data quality checks on data pipelines☆256Updated 3 years ago
- A docker image with a pre-configured Hive Metastore and a Spark ThriftServer☆19Updated 6 years ago