datumo / dataset-logger

Related projects ⓘ

Alternatives and complementary repositories for dataset-logger

• RTBHOUSE / avro-fastserde
  Fast Apache Avro serialization/deserialization library
  ☆43Updated 4 years ago
• bartosz25 / spark-playground
  Code snippets used in demos recorded for the blog.
  ☆29Updated last month
• 10xfuturetechnologies / kafka-connect-iceberg
  Kafka Connector for Iceberg tables
  ☆16Updated last year
• zalando-incubator / spark-json-schema
  JSON schema parser for Apache Spark
  ☆81Updated 2 years ago
• wjur / avsc2avdl
  Avro Schema to Avro IDL converter
  ☆16Updated 3 years ago
• CoxAutomotiveDataSolutions / waimak
  Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.
  ☆75Updated 6 months ago
• embeddedkafka / embedded-kafka-schema-registry
  A library that provides in-memory instances of both Kafka and Confluent Schema Registry to run your tests against.
  ☆111Updated this week
• zeotap / spark-property-tests
  Write property based tests easily on spark dataframes
  ☆19Updated 10 months ago
• linkedin / avro-util
  Collection of utilities to allow writing java code that operates across a wide range of avro versions.
  ☆76Updated this week
• holdenk / spark-flowchart
  Flowchart for debugging Spark applications
  ☆101Updated last month
• hortonworks-spark / spark-schema-registry
  Schema Registry integration for Apache Spark
  ☆39Updated 2 years ago
• spotify / scio.g8
  A Giter8 template for scio
  ☆30Updated this week
• AbsaOSS / atum
  A dynamic data completeness and accuracy library at enterprise scale for Apache Spark
  ☆30Updated 2 weeks ago
• funkyminds / cleanframes
  type-class based data cleansing library for Apache Spark SQL
  ☆79Updated 5 years ago
• HeartSaVioR / spark-state-tools
  Spark Structured Streaming State Tools
  ☆34Updated 4 years ago
• allegro / bigflow
  A Python framework for data processing on GCP.
  ☆117Updated last month
• AbsaOSS / spark-hats
  Nested array transformation helper extensions for Apache Spark
  ☆36Updated last year
• godatadriven / scala-spark-application
  ☆33Updated 6 years ago
• ExpediaGroup / hello-streams
  hello-streams :: Introducing the stream-first mindset
  ☆17Updated 8 months ago
• swoop-inc / spark-records
  Bulletproof Apache Spark jobs with fast root cause analysis of failures.
  ☆72Updated 3 years ago
• lensesio / secret-provider
  Open Source Secret Provider plugin for the Kafka Connect framework
  ☆45Updated 4 months ago
• holdenk / spark-upgrade
  Magic to help Spark pipelines upgrade
  ☆34Updated last month
• yaooqinn / itachi
  A library that brings useful functions from various modern database management systems to Apache Spark
  ☆56Updated last year
• MrPowers / spark-slack
  Speak Slack notifications and process Slack slash commands
  ☆14Updated 5 years ago
• ing-bank / scruid
  Scala + Druid: Scruid. A library that allows you to compose queries in Scala, and parse the result back into typesafe classes.
  ☆115Updated 3 years ago
• holdenk / high-performance-spark-examples
  Examples for High Performance Spark
  ☆15Updated 2 weeks ago
• KeithSSmith / spark-compaction
  File compaction tool that runs on top of the Spark framework.
  ☆59Updated 5 years ago
• aseigneurin / kafka-streams-scala
  Thin Scala wrapper for the Kafka Streams API
  ☆50Updated 6 years ago