yamrcraft/etl-light external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

yamrcraft/etl-light

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/yamrcraft/etl-light)

Close

yamrcraft / etl-lightView on GitHubMore

• External links
• Readme badge

A light Kafka to HDFS/S3 ETL library based on Apache Spark

☆40Jun 29, 2017Updated 8 years ago

Alternatives and similar repositories for etl-light

Users that are interested in etl-light are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• konrads / spark-etl

  View on GitHub
  Set of ETL utils for Spark
  ☆15May 4, 2020Updated 5 years ago
• vngrs / spark-etl

  View on GitHub
  Apache Spark based ETL Engine
  ☆71Oct 18, 2016Updated 9 years ago
• ycloudnet / ya100

  View on GitHub
  一个比Spark-Parquet还快5~100倍的存储格式
  ☆12Feb 22, 2016Updated 10 years ago
• XavientInformationSystems / Data-Ingestion-Platform

  View on GitHub
  ☆50Feb 11, 2020Updated 6 years ago
• dk-stationery / stationery-ink

  View on GitHub
  Distributed SQL base Realtime Streaming Computation Framework On Apache Storm, Spark
  ☆12Mar 14, 2016Updated 10 years ago
• Bare Metal GPUs on DigitalOcean Gradient AI • Ad
  Purpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
• wuworker / netty-proxy

  View on GitHub
  基于netty实现代理服务器
  ☆11Nov 17, 2019Updated 6 years ago
• anjuke / romar

  View on GitHub
  Recommendation Web Service
  ☆17Apr 17, 2013Updated 12 years ago
• wykingfly / ElasticSearch-SQL

  View on GitHub
  该项目主要是为了熟悉sql的人员能够很方便的进行elasticsearch数据的查询，降低学习成本。
  ☆47Jan 13, 2015Updated 11 years ago
• bluebreezecf / SparkJobServerClient

  View on GitHub
  Java Client of the Spark Job Server implementing the arranged Rest APIs
  ☆51Jun 4, 2021Updated 4 years ago
• johanprinsloo / akka-micro-dag

  View on GitHub
  Quick Akka Micro Dag Prototype
  ☆13Apr 8, 2016Updated 9 years ago
• CodeRayZhang / Spark-Example

  View on GitHub
  Spark1.6和spark2.2的示例，包含kafka,flume,structuredstreaming,jedis,elasticsearch,mysql,dataframe
  ☆15Jan 28, 2018Updated 8 years ago
• uber / uberscriptquery

  View on GitHub
  UberScriptQuery, a SQL-like DSL to make writing Spark jobs super easy
  ☆64Dec 17, 2023Updated 2 years ago
• cpbaranwal / Avro-SparkStreaming-Kafka

  View on GitHub
  Code for processing AVRO data in Spark Streaming + Kafka (DirectKafka approach with custom offset management in zookeeper)
  ☆29Sep 9, 2016Updated 9 years ago
• dianping / storm-util

  View on GitHub
  ☆18Apr 23, 2015Updated 10 years ago
• Virtual machines for every use case on DigitalOcean • Ad
  Get dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
• RedisLabs / ReSearch

  View on GitHub
  Redis search and indexing in Java
  ☆16Sep 26, 2016Updated 9 years ago
• hammerlab / spark-tests

  View on GitHub
  Utilities for writing tests that use Apache Spark.
  ☆24Dec 29, 2018Updated 7 years ago
• thunderain-project / thunderain

  View on GitHub
  A Real-Time Analytical Processing (RTAP) example using Spark/Shark
  ☆51Feb 21, 2014Updated 12 years ago
• chenzhenyang / aquila

  View on GitHub
  迁移工具，目标是Oracle，MySQL，SqlServer到PostgreSQL的单项迁移，PostgreSQL和大数据平台Hive，Hbase，Impala等的双向迁移。
  ☆10Dec 3, 2014Updated 11 years ago
• leonLMR / presto-es

  View on GitHub
  presto's elasticsearch connector
  ☆11Dec 7, 2016Updated 9 years ago
• vspiewak / twitter-sentiment-analysis

  View on GitHub
  Streaming tweets with spark, language detection & sentiment analysis, dashboard with Kibana
  ☆105Dec 21, 2015Updated 10 years ago
• cpbaranwal / Spark-Streaming-DirectKafka-Examples

  View on GitHub
  DirectKafka examples for Spark Streaming : 1. with checkpointing 2. Custom offset management
  ☆60Sep 9, 2016Updated 9 years ago
• BenFradet / spark-kafka-writer

  View on GitHub
  Write your Spark data to Kafka seamlessly
  ☆173Jul 10, 2024Updated last year
• wypb / Spark-Flink-Meetup-6-Hangzhou

  View on GitHub
  杭州第六次 Spark & Flink Meetup
  ☆30May 14, 2018Updated 7 years ago
• Managed Database hosting by DigitalOcean • Ad
  PostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
• xiam / resp

  View on GitHub
  RESP (REdis Serialization Protocol) encoder and decoder.
  ☆19Dec 6, 2015Updated 10 years ago
• duoan / codes-scratch-crawler

  View on GitHub
  读书笔记《自己动手写网络爬虫》，自己敲的代码。主要记录了网络爬虫的基本实现，网页去重的算法，网页指纹算法，文本信息挖掘
  ☆47Jan 9, 2015Updated 11 years ago
• pdaodao / fiflow

  View on GitHub
  flink-sql 在 flink 上运行 sql 和 构建数据流的平台 基于 apache flink 1.10.0
  ☆113Jun 21, 2022Updated 3 years ago
• bernhard-42 / Spark-ETL-Atlas

  View on GitHub
  A small project to show how to add lineage to Atlas when using Spark as ETL tool
  ☆12Nov 29, 2016Updated 9 years ago
• realxujiang / storm-kafka-examples

  View on GitHub
  storm kafka hdfs examples
  ☆21Nov 28, 2016Updated 9 years ago
• ndolgov / experiments

  View on GitHub
  Code examples for my blog posts
  ☆22Nov 7, 2018Updated 7 years ago
• JasonWiki / dw_etl

  View on GitHub
  dw etl 工具 mysql 增量、全量抽取 to hive. 合并 hive 数据表, 等数据平台清洗工具
  ☆10Dec 21, 2016Updated 9 years ago
• zengxiaosen / flinkMultiStreamOptimization

  View on GitHub
  优化flink的多流操作（例如join），优化点不限于数据丢失问题，以及性能问题
  ☆11Apr 8, 2019Updated 6 years ago
• markgrover / spark-secure-kafka-app

  View on GitHub
  Sample Spark Streaming application for secure consumption from Kafka
  ☆33Jun 19, 2017Updated 8 years ago
• Wordpress hosting with auto-scaling on Cloudways • Ad
  Fully Managed hosting built for WordPress-powered businesses that need reliable, auto-scalable hosting. Cloudways SafeUpdates now available.
• ml-hongkong / chatbot

  View on GitHub
  Seq2Seq Chatbot with attention mechanism
  ☆19Apr 27, 2017Updated 8 years ago
• jeoffreylim / maelstrom

  View on GitHub
  Maelstrom is an open source Kafka integration with Spark that is designed to be developer friendly, high performance (millisecond stream …
  ☆22Feb 6, 2017Updated 9 years ago
• ExpediaGroup / datasqueeze

  View on GitHub
  Hadoop utility to compact small files
  ☆18Feb 16, 2026Updated last month
• mispecto / realtime-dashboard-example

  View on GitHub
  This is a real-time dashboard example using Spark Streaming and Node.js
  ☆26Dec 17, 2025Updated 3 months ago
• sujee81 / SparkApps

  View on GitHub
  Apache Spark applications
  ☆70Dec 17, 2017Updated 8 years ago
• fbascheper / kafka-connect-telegram

  View on GitHub
  Kafka-connect telegram connector
  ☆16Nov 21, 2025Updated 4 months ago
• OopsOutOfMemory / spark-sql-hbase

  View on GitHub
  A Spark SQL HBase connector
  ☆29May 4, 2015Updated 10 years ago