针对datax进行2次开发，实现data 以rpc的方式传递json配置调用推数服务，同时修复datax多处bug。项目中也引入nacos作为服务的配置中心和注册中心； 同时项目内扩展了kafkawriter，rabbitmqwriter，esreader，hivereader。增强了hdfs插件，支持分区表推送，支持动态参数传递（例如时间实现自增式抽取）。具体使用方式可以参照example模块。目前该服务已经稳定服务某上市公司半年，累计总任务数100+ ，日推送数据过10亿。具体如何使用，如何做插件开发以及datax底层原理，请关注https://blog.csdn.net/xiaoyao1999hn
☆70Dec 14, 2022Updated 3 years ago
Alternatives and similar repositories for datax-service
Users that are interested in datax-service are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 将datax改成一个java服务包，支持单jvm同时运行多个实例，整合调度代码，提供api调用接口，可以直接集成到已有服务框架。☆38Dec 6, 2022Updated 3 years ago
- 基于阿里Datax改版web datax ,支持管理平台与restful风格API☆29Dec 6, 2022Updated 3 years ago
- skycloud-base 运维管理平台前端页面☆15Dec 10, 2022Updated 3 years ago
- DataX分布式集群与负载均衡、任务执行/统计，基于DataX的通用数据同步微服务，一个Restful接口搞定所有 通用数据同步☆44Jan 18, 2021Updated 5 years ago
- ☆26Oct 23, 2019Updated 6 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- datax的elasticsearch插件，主要是reader插件，writer插件官网已经实现了。适用于es7.x☆10Mar 6, 2021Updated 5 years ago
- 使用springboot启动datax，方便以web方式使用☆225Jan 11, 2023Updated 3 years ago
- 早该知道的一种更高级的 transformer 自定义实现方法。☆30Mar 28, 2019Updated 7 years ago
- HiveReader for alibaba DataX☆17Jul 23, 2023Updated 3 years ago
- datax web。datax中的web配置界面没有集成在一起开源出来，此为web端配置项目。☆100Mar 19, 2019Updated 7 years ago
- 流程化 机器学习框架 基于 scala java语言 ,一站式自动机器学习平台 ,主要包括数据分析 特征工程 ，机器模型，自动部署，超参数优化，模型自动优化，自动扩容分配创建功能，类似第四范式、阿里PAI平台、google autoMl、亚马逊SageMaker☆68Aug 1, 2018Updated 7 years ago
- springboot auto xss☆11May 23, 2018Updated 8 years ago
- 元数据管理系统☆17Jan 20, 2018Updated 8 years ago
- springultron Spring Boot multi module common library☆11Jun 25, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 记录整理日常java安全分析代码☆14Dec 25, 2018Updated 7 years ago
- 为DataX(https://github.com/alibaba/DataX) 提供远程多语言调用（ThriftServer，HttpServer） 分布式运行（DataX on YARN） 功能☆144Jul 15, 2026Updated last week
- DataX集成可视化页面，选择数据源即可一键生成数据同步任务，支持RDBMS、Hive、HBase、ClickHouse、MongoDB等数据源，批量创建RDBMS数据同步任务，集成开源调度系统，支持分布式、增量同步数据、实时查看运行日志、监控执行器资源、KILL运行进程、…☆6,018Jun 2, 2024Updated 2 years ago
- A library enabling DAG structuring of data processing programs such as ETLs☆18Jul 19, 2026Updated last week
- 数据血缘，Hive/Sqoop/HBase/Spark等，发送到kafka后，解析处理使用neo4j生成血缘☆86Aug 6, 2021Updated 4 years ago
- elasticsearch reader and writer plugin for datax☆39Aug 31, 2017Updated 8 years ago
- datax写redis插件，支持异构数据源导入string、list、hash类型导redis☆19Sep 29, 2021Updated 4 years ago
- Elasticsearch_7 基本操作、嵌套查询、MultiSearchTemplateRequest，rangeQuery，父子关系的建立，更新及查询，分页 等☆11Nov 22, 2023Updated 2 years ago
- Open data platform based on Kubernetes. Scaleph supports SeaTunnel、Flink and Doris backended by SeaTunnel on Flink engine、Flink Kubernete…☆395Dec 17, 2025Updated 7 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ClickHouse connector both for PrestoSQL and Trino☆18Mar 3, 2021Updated 5 years ago
- 电商用户行为分析大数据平台☆34Nov 13, 2019Updated 6 years ago
- ActorCloud Documentation☆13Oct 5, 2022Updated 3 years ago
- flink sql☆11Jun 21, 2022Updated 4 years ago
- Exchangis is a lightweight,highly extensible data exchange platform that supports data transmission between structured and unstructured h…☆461Oct 28, 2025Updated 9 months ago
- Full Database Migration Tool based on Alibaba DataX 3.0☆103Oct 28, 2019Updated 6 years ago
- 基于jeecg-boot封装的spring-boot-starter, 可用于快速开发，享受jeecg-boot的所有功能，且升级内核业务无感知！☆23Nov 8, 2020Updated 5 years ago
- ranger and ldap demo☆10Apr 14, 2023Updated 3 years ago
- Addax Admin is a web-based management console for Addax ETL jobs, offering task scheduling, monitoring, and data management with a Vue 3 …☆29Jul 21, 2026Updated last week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI☆1,314Updated this week
- OLAMI API Java client library and sample codes☆10May 20, 2022Updated 4 years ago
- dataX redis writer plugin☆12Jul 13, 2017Updated 9 years ago
- Plot live-stats as graph from ApacheSpark application using Lightning-viz☆18Jul 3, 2017Updated 9 years ago
- HiveWriter for alibaba DataX☆14Sep 25, 2024Updated last year
- for iot-gateway device drive(device/ammeter下有完整DLT645-2007驱动和例子)☆10Dec 8, 2022Updated 3 years ago
- 此微服务包括分布式认证中心（jwt+redis），Eureka 集群，客户端负载均衡 turbin微服务监控。Hystrix服务熔断机制 请求限流。以及数据同步DATAX (https://github.com/alibaba/DataX)，☆25Apr 2, 2018Updated 8 years ago