hunters-ai / spark-adaptive-file-connectorLinks
Adaptive File Source Connector for Spark, optimised for reading from object stores
☆15Updated 3 years ago
Alternatives and similar repositories for spark-adaptive-file-connector
Users that are interested in spark-adaptive-file-connector are comparing it to the libraries listed below
Sorting:
- Maven plugin for generating Scala case classes and ADTs from Apache Avro schemas, datafiles, and protocols☆10Updated 2 years ago
- Apache Amaterasu☆56Updated 6 years ago
- Scala + Druid: Scruid. A library that allows you to compose queries in Scala, and parse the result back into typesafe classes.☆115Updated 4 years ago
- Big Data Toolkit for the JVM☆145Updated 4 years ago
- Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.☆112Updated 5 years ago
- low-level helpers for Apache Spark libraries and tests☆16Updated 6 years ago
- Schema Registry integration for Apache Spark☆40Updated 2 years ago
- A framework for creating composable and pluggable data processing pipelines using Apache Spark, and running them on a cluster.☆47Updated 9 years ago
- A dynamic data completeness and accuracy library at enterprise scale for Apache Spark☆29Updated 11 months ago
- A collection of Apache Parquet add-on modules☆30Updated last month
- Used to generate mock Avro data☆15Updated 7 years ago
- Data-Driven Spark allows quick data exploration based on Apache Spark.☆29Updated 8 years ago
- Deriving Spark DataFrame schemas from case classes☆44Updated last year
- something to help you spark☆64Updated 6 years ago
- functionstest☆33Updated 8 years ago
- Thin Scala wrapper for the Kafka Streams API☆49Updated 7 years ago
- ☆50Updated 5 years ago
- An embedded job scheduler.☆116Updated last year
- Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.☆15Updated 7 years ago
- Schedoscope is a scheduling framework for painfree agile development, testing, (re)loading, and monitoring of your datahub, lake, or what…☆96Updated 5 years ago
- Spark cloud integration: tests, cloud committers and more☆20Updated 8 months ago
- Spark package to "plug" holes in data using SQL based rules ⚡️ 🔌☆29Updated 5 years ago
- type-class based data cleansing library for Apache Spark SQL☆78Updated 6 years ago
- Fast, memory-efficient, minimal-serialization, binary data vectors for Scala and other languages☆67Updated 7 years ago
- A quotation-based Scala DSL for scalable data analysis.☆63Updated 3 years ago
- Spark Structured Streaming State Tools☆34Updated 5 years ago
- Library for organizing batch processing pipelines in Apache Spark☆42Updated 8 years ago
- Hadoop-Unit is a project which allow testing projects which need hadoop ecosysteme like kafka, solr, hdfs, hive, hbase, ...☆52Updated last week
- Repository for advanced unit-testing with embedded kafka services☆25Updated 6 years ago
- Scala library for Reactive streaming Microservices, CQRS, Event Sourcing, Event Logging, & message-driven apps.☆23Updated 7 years ago