yamrcraft/etl-light external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

yamrcraft/etl-light

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/yamrcraft/etl-light)

Close

yamrcraft / etl-lightView on GitHubMore

• External links
• Readme badge

A light Kafka to HDFS/S3 ETL library based on Apache Spark

☆40Jun 29, 2017Updated 8 years ago

Alternatives and similar repositories for etl-light

Users that are interested in etl-light are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• konrads / spark-etl

  View on GitHub
  Set of ETL utils for Spark
  ☆15May 4, 2020Updated 6 years ago
• vngrs / spark-etl

  View on GitHub
  Apache Spark based ETL Engine
  ☆71Oct 18, 2016Updated 9 years ago
• ycloudnet / ya100

  View on GitHub
  一个比Spark-Parquet还快5~100倍的存储格式
  ☆12Feb 22, 2016Updated 10 years ago
• xavient / Data-Ingestion-Platform

  View on GitHub
  ☆50May 21, 2026Updated last week
• dk-stationery / stationery-ink

  View on GitHub
  Distributed SQL base Realtime Streaming Computation Framework On Apache Storm, Spark
  ☆12Mar 14, 2016Updated 10 years ago
• Deploy on Railway without the complexity - Free Credits Offer • Ad
  Connect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
• shengjk / flinksql-platform

  View on GitHub
  flinksql-platform
  ☆19Mar 22, 2021Updated 5 years ago
• wuworker / netty-proxy

  View on GitHub
  基于netty实现代理服务器
  ☆11Nov 17, 2019Updated 6 years ago
• pranab / chombo

  View on GitHub
  Big Data ETL and Utilities for Hadoop Map Reduce, Spark and Storm
  ☆106Jan 22, 2024Updated 2 years ago
• wykingfly / ElasticSearch-SQL

  View on GitHub
  该项目主要是为了熟悉sql的人员能够很方便的进行elasticsearch数据的查询，降低学习成本。
  ☆47Jan 13, 2015Updated 11 years ago
• icgc-dcc / dcc-release

  View on GitHub
  Second generation of the ICGC DCC release ETL built on Spark
  ☆10Apr 8, 2019Updated 7 years ago
• bluebreezecf / SparkJobServerClient

  View on GitHub
  Java Client of the Spark Job Server implementing the arranged Rest APIs
  ☆51Jun 4, 2021Updated 4 years ago
• johanprinsloo / akka-micro-dag

  View on GitHub
  Quick Akka Micro Dag Prototype
  ☆13Apr 8, 2016Updated 10 years ago
• uber / uberscriptquery

  View on GitHub
  UberScriptQuery, a SQL-like DSL to make writing Spark jobs super easy
  ☆65Dec 17, 2023Updated 2 years ago
• CodeRayZhang / Spark-Example

  View on GitHub
  Spark1.6和spark2.2的示例，包含kafka,flume,structuredstreaming,jedis,elasticsearch,mysql,dataframe
  ☆15Jan 28, 2018Updated 8 years ago
• Deploy to Railway using AI coding agents - Free Credits Offer • Ad
  Use Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
• cpbaranwal / Avro-SparkStreaming-Kafka

  View on GitHub
  Code for processing AVRO data in Spark Streaming + Kafka (DirectKafka approach with custom offset management in zookeeper)
  ☆29Sep 9, 2016Updated 9 years ago
• dianping / storm-util

  View on GitHub
  ☆18Apr 23, 2015Updated 11 years ago
• RedisLabs / ReSearch

  View on GitHub
  Redis search and indexing in Java
  ☆16Sep 26, 2016Updated 9 years ago
• bylee5 / calcite-elasticsearch

  View on GitHub
  ☆14Oct 5, 2022Updated 3 years ago
• thunderain-project / thunderain

  View on GitHub
  A Real-Time Analytical Processing (RTAP) example using Spark/Shark
  ☆51Feb 21, 2014Updated 12 years ago
• leonLMR / presto-es

  View on GitHub
  presto's elasticsearch connector
  ☆11Dec 7, 2016Updated 9 years ago
• chenzhenyang / aquila

  View on GitHub
  迁移工具，目标是Oracle，MySQL，SqlServer到PostgreSQL的单项迁移，PostgreSQL和大数据平台Hive，Hbase，Impala等的双向迁移。
  ☆10Dec 3, 2014Updated 11 years ago
• cpbaranwal / Spark-Streaming-DirectKafka-Examples

  View on GitHub
  DirectKafka examples for Spark Streaming : 1. with checkpointing 2. Custom offset management
  ☆60Sep 9, 2016Updated 9 years ago
• BenFradet / spark-kafka-writer

  View on GitHub
  Write your Spark data to Kafka seamlessly
  ☆174Jul 10, 2024Updated last year
• End-to-end encrypted email - Proton Mail • Ad
  Special offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
• Eterland / eterland-public-smart-contracts

  View on GitHub
  contains non confidential smart contracts from eterland
  ☆16Jun 8, 2023Updated 2 years ago
• wypb / Spark-Flink-Meetup-6-Hangzhou

  View on GitHub
  杭州第六次 Spark & Flink Meetup
  ☆30May 14, 2018Updated 8 years ago
• xiam / resp

  View on GitHub
  RESP (REdis Serialization Protocol) encoder and decoder.
  ☆19Dec 6, 2015Updated 10 years ago
• duoan / codes-scratch-crawler

  View on GitHub
  读书笔记《自己动手写网络爬虫》，自己敲的代码。主要记录了网络爬虫的基本实现，网页去重的算法，网页指纹算法，文本信息挖掘
  ☆47Jan 9, 2015Updated 11 years ago
• pdaodao / fiflow

  View on GitHub
  flink-sql 在 flink 上运行 sql 和 构建数据流的平台 基于 apache flink 1.10.0
  ☆113Jun 21, 2022Updated 3 years ago
• Stratio / sparta

  View on GitHub
  Real Time Analytics and Data Pipelines based on Spark Streaming
  ☆531Oct 24, 2019Updated 6 years ago
• realxujiang / storm-kafka-examples

  View on GitHub
  storm kafka hdfs examples
  ☆21Nov 28, 2016Updated 9 years ago
• bernhard-42 / Spark-ETL-Atlas

  View on GitHub
  A small project to show how to add lineage to Atlas when using Spark as ETL tool
  ☆12Nov 29, 2016Updated 9 years ago
• ndolgov / experiments

  View on GitHub
  Code examples for my blog posts
  ☆22Nov 7, 2018Updated 7 years ago
• Managed hosting for WordPress and PHP on Cloudways • Ad
  Managed hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
• JasonWiki / dw_etl

  View on GitHub
  dw etl 工具 mysql 增量、全量抽取 to hive. 合并 hive 数据表, 等数据平台清洗工具
  ☆10Dec 21, 2016Updated 9 years ago
• zengxiaosen / flinkMultiStreamOptimization

  View on GitHub
  优化flink的多流操作（例如join），优化点不限于数据丢失问题，以及性能问题
  ☆11Apr 8, 2019Updated 7 years ago
• markgrover / spark-secure-kafka-app

  View on GitHub
  Sample Spark Streaming application for secure consumption from Kafka
  ☆33Jun 19, 2017Updated 8 years ago
• ml-hongkong / chatbot

  View on GitHub
  Seq2Seq Chatbot with attention mechanism
  ☆19Apr 27, 2017Updated 9 years ago
• jeoffreylim / maelstrom

  View on GitHub
  Maelstrom is an open source Kafka integration with Spark that is designed to be developer friendly, high performance (millisecond stream …
  ☆22Feb 6, 2017Updated 9 years ago
• ExpediaGroup / datasqueeze

  View on GitHub
  Hadoop utility to compact small files
  ☆18Feb 16, 2026Updated 3 months ago
• mispecto / realtime-dashboard-example

  View on GitHub
  This is a real-time dashboard example using Spark Streaming and Node.js
  ☆26Dec 17, 2025Updated 5 months ago