ttauveron / k8s-big-data-experiments
Experiments produced during an end of studies project (ETS, H2018)
☆15Updated 6 years ago
Alternatives and similar repositories for k8s-big-data-experiments
Users that are interested in k8s-big-data-experiments are comparing it to the libraries listed below
Sorting:
- Scala SDK for working with Snowplow enriched events in Spark, AWS Lambda, Flink et al.☆21Updated 6 months ago
- Data validation library for PySpark 3.0.0☆33Updated 2 years ago
- Spark on Kubernetes infrastructure Docker images repo☆37Updated 2 years ago
- ETLy is an add-on dashboard service on top of Apache Airflow.☆68Updated last year
- A plugin to Apache Airflow to allow you to run Spark Submit Commands as an Operator☆73Updated 5 years ago
- Schema Registry integration for Apache Spark☆40Updated 2 years ago
- A facebook for data☆26Updated 5 years ago
- Cloud Spanner Connector for Apache Spark☆17Updated 4 months ago
- Oozie Workflow to Airflow DAGs migration tool☆87Updated 2 months ago
- A library that brings useful functions from various modern database management systems to Apache Spark☆58Updated last year
- ☆10Updated 6 years ago
- ☆26Updated 9 years ago
- REST-like API exposing Airflow data and operations☆61Updated 6 years ago
- The dbt-spark-livy adapter allows you to use dbt along with Apache Spark, by connecting via Apache Livy☆12Updated 2 years ago
- ☆63Updated 5 years ago
- JSON schema parser for Apache Spark☆81Updated 2 years ago
- Apache (Py)Spark type annotations (stub files).☆117Updated 2 years ago
- How to manage Slowly Changing Dimensions with Apache Hive☆55Updated 5 years ago
- Bare minimal Airflow on Kubernetes (Local, EKS, AKS)☆53Updated 5 years ago
- Airflow declarative DAGs via YAML☆132Updated last year
- Quickly get a kubernetes executor airflow environment provisioned on GKE. Azure Kubernetes Service instructions included also as are inst…☆36Updated 4 years ago
- Avro Schema Shredder is a REST API that enables storage of Avro Schemas in Apache Atlas. This API enables an organization to use Apache A…☆13Updated 8 years ago
- A Python client for Apache Livy, enabling use of remote Apache Spark clusters.☆70Updated 3 years ago
- Task Metrics Explorer☆13Updated 6 years ago
- Magic to help Spark pipelines upgrade☆35Updated 7 months ago
- Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple…☆26Updated 3 years ago
- PySpark for ETL jobs including lineage to Apache Atlas in one script via code inspection☆18Updated 8 years ago
- A Spark-based data comparison tool at scale which facilitates software development engineers to compare a plethora of pair combinations o…☆51Updated last year
- Python client for Marquez☆12Updated 4 years ago
- File compaction tool that runs on top of the Spark framework.☆59Updated 6 years ago