fingltd / 4mc
4mc - splittable lz4 and zstd in hadoop/spark/flink
☆109Updated 2 years ago
Alternatives and similar repositories for 4mc
Users that are interested in 4mc are comparing it to the libraries listed below
Sorting:
- Spark SQL index for Parquet tables☆134Updated 4 years ago
- Cache File System optimized for columnar formats and object stores☆182Updated 2 years ago
- Hadoop output committers for S3☆109Updated 4 years ago
- Stocator is high performing connector to object storage for Apache Spark, achieving performance by leveraging object storage semantics.☆114Updated last year
- A library to expose more of Apache Spark's metrics system☆146Updated 5 years ago
- Enabling Spark Optimization through Cross-stack Monitoring and Visualization☆47Updated 7 years ago
- Lightweight proxy to expose the UI of an Apache Spark cluster that is behind a firewall☆98Updated 4 years ago
- Albis: High-Performance File Format for Big Data Systems☆21Updated 6 years ago
- Enabling queries on compressed data.☆279Updated last year
- Sketch adaptors for Hive.☆50Updated 2 months ago
- A Tez dev-setup for HDP2 sandbox☆21Updated 2 years ago
- Easy CPU Profiling for Apache Spark applications☆45Updated 4 years ago
- Druid indexing plugin for using Spark in batch jobs☆101Updated 3 years ago
- Secondary sort and streaming reduce for Apache Spark☆78Updated last year
- Spark RDD with Lucene's query and entity linkage capabilities☆127Updated last month
- Tools for working with parquet, impala, and hive☆134Updated 4 years ago
- Hadoop Profiler, or hprofiler, is a tool which is able to analyze on- and off-CPU workloads on distributed computing environments.☆24Updated 8 years ago
- Drizzle integration with Apache Spark☆120Updated 6 years ago
- TPC-DS Kit for Impala☆170Updated 11 months ago
- Read SparkSQL parquet file as RDD[Protobuf]☆93Updated 6 years ago
- Quark is a data virtualization engine over analytic databases.☆97Updated 7 years ago
- Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange☆127Updated 4 months ago
- Profiler for large-scale distributed java applications (Spark, Scalding, MapReduce, Hive,...) on YARN.☆126Updated 6 years ago
- A tool for scale and performance testing of HDFS with a specific focus on the NameNode.☆131Updated last year
- The SpliceSQL Engine☆169Updated last year
- A High Performance Cluster Consumer for Kafka that creates Avro (boom) files in Hadoop in time based directory paths☆42Updated 8 years ago
- Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive☆185Updated 2 years ago
- Splittable Gzip codec for Hadoop☆70Updated last month
- Remedy small files by combining them into larger ones.☆193Updated 2 years ago
- Spark metrics related custom classes and sinks (e.g. Prometheus)☆181Updated 2 years ago