MrDataPsycho / data-pipelines-in-rust
Data pipeline example written in Rust with Polars and DataFusion DataFrame package
☆40Updated last year
Alternatives and similar repositories for data-pipelines-in-rust:
Users that are interested in data-pipelines-in-rust are comparing it to the libraries listed below
- ☆22Updated 3 years ago
- Rust SDK for Apache Avro - a data serialization system.☆35Updated this week
- HDFS based on Java implementation as a remote ObjectStore for DataFusion☆9Updated 11 months ago
- ☆21Updated 8 months ago
- Experimental support for serializing DataFusion plans using substrait☆44Updated 2 years ago
- Apache Arrow Ballista Python bindings☆36Updated 11 months ago
- Robust data transformation tool using SQL☆20Updated 2 years ago
- A presto/trino client library written in rust.☆40Updated last month
- A text embedding extension for the Polars Dataframe library.☆24Updated 2 months ago
- JSON support for DataFusion (unofficial)☆32Updated 2 weeks ago
- Python binding for DataFusion☆59Updated 2 years ago
- An opinionated and batteries included DataFusion implementation.☆128Updated this week
- Tantivy directory implementation backed by object_store☆29Updated last year
- µWheel DataFusion Optimizer for speeding up time-based analytics☆30Updated 2 weeks ago
- dbt-databend adapter plugin☆10Updated 8 months ago
- Fill Apache Arrow record batches from an ODBC data source in Rust.☆60Updated 3 weeks ago
- ☆83Updated last month
- Rust crate for Substrait: Cross-Language Serialization for Relational Algebra☆61Updated this week
- Automated Machine Learning in Rust 🦀☆33Updated 3 weeks ago
- Framework to build data pipelines declaratively☆49Updated last week
- Rust FFI example binding for chDB, an embedded SQL Engine powered by ClickHouse☆31Updated 5 months ago
- Bigtable data source for Apache Arrow DataFusion☆23Updated 2 years ago
- S3 as an ObjectStore for DataFusion☆60Updated last year
- DataFusion TableProviders for reading data from other systems☆79Updated this week
- HDFS Native Client in Rust via HDFS C API libhdfs☆35Updated this week
- Open, Multi-modal Catalog for Data & AI, written in Rust☆76Updated 4 months ago
- Generated Rust of Apache Arrow spec☆16Updated last year
- WASM bindings for DataFusion☆16Updated last month
- A high-performance RAG indexing pipeline implemented in Rust using LanceDB and Candle☆13Updated 5 months ago
- Rust lib to read from Apache ORC☆18Updated last year