hunters-ai / spark-adaptive-file-connectorLinks
Adaptive File Source Connector for Spark, optimised for reading from object stores
☆15Updated 2 years ago
Alternatives and similar repositories for spark-adaptive-file-connector
Users that are interested in spark-adaptive-file-connector are comparing it to the libraries listed below
Sorting:
- Maven plugin for generating Scala case classes and ADTs from Apache Avro schemas, datafiles, and protocols☆10Updated last year
- Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.☆112Updated 5 years ago
- Scala + Druid: Scruid. A library that allows you to compose queries in Scala, and parse the result back into typesafe classes.☆115Updated 4 years ago
- Deriving Spark DataFrame schemas from case classes☆44Updated last year
- Big Data Toolkit for the JVM☆146Updated 4 years ago
- Apache Amaterasu☆56Updated 5 years ago
- ☆50Updated 4 years ago
- A framework for creating composable and pluggable data processing pipelines using Apache Spark, and running them on a cluster.☆47Updated 8 years ago
- Writing application logic for Spark jobs that can be unit-tested without a SparkContext☆77Updated 6 years ago
- Spark cloud integration: tests, cloud committers and more☆19Updated 5 months ago
- low-level helpers for Apache Spark libraries and tests☆16Updated 6 years ago
- Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.☆15Updated 7 years ago
- A collection of Apache Parquet add-on modules☆30Updated 2 weeks ago
- ☆45Updated 5 years ago
- Schema Registry integration for Apache Spark☆40Updated 2 years ago
- A quotation-based Scala DSL for scalable data analysis.☆63Updated 3 years ago
- Spark Structured Streaming State Tools☆34Updated 5 years ago
- A dynamic data completeness and accuracy library at enterprise scale for Apache Spark☆29Updated 8 months ago
- ☆36Updated 2 years ago
- An embedded job scheduler.☆115Updated 11 months ago
- Sbt plugin for avro4s☆20Updated 7 years ago
- An SBT plugin for automatically calling Avro code generation and a thin scala wrapper for reading and writing Avro files☆22Updated 7 years ago
- Data-Driven Spark allows quick data exploration based on Apache Spark.☆28Updated 8 years ago
- A library that provides in-memory instances of both Kafka and Confluent Schema Registry to run your tests against.☆115Updated 2 weeks ago
- An easy way to deploy your Akka services to a distributed environment.☆30Updated 2 years ago
- Thin Scala wrapper for the Kafka Streams API☆49Updated 7 years ago
- Model complex data transformation pipelines easily☆46Updated 2 years ago
- Data quality tools for Big Data☆19Updated 5 years ago
- Lego bricks to build Apache Kafka serializers and deserializers☆120Updated last year
- Scala Mison implementation☆15Updated 6 years ago