lancedb / lance
Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
☆4,227Updated this week
Alternatives and similar repositories for lance:
Users that are interested in lance are comparing it to the libraries listed below
- Distributed data engine for Python/SQL designed for the cloud, powered by Rust☆2,573Updated this week
- Apache DataFusion Ballista Distributed Query Engine☆1,674Updated last week
- Distributed stream processing engine in Rust☆4,021Updated this week
- Developer-friendly, serverless vector database for AI applications. Easily add long-term memory to your LLM apps!☆5,750Updated this week
- Apache DataFusion SQL Query Engine☆6,832Updated this week
- A native Rust library for Delta Lake, with bindings into Python☆2,588Updated this week
- Making data lake work for time series☆1,153Updated 6 months ago
- A transactional, relational-graph-vector database that uses Datalog for query. The hippocampus for AI!☆3,544Updated 2 months ago
- Apache OpenDAL: One Layer, All Storage.☆3,874Updated this week
- Fastest library to load data from DB to DataFrames in Rust and Python☆2,135Updated last week
- Official Rust implementation of Apache Arrow☆2,801Updated this week
- Postgres with GPUs for ML/AI apps.☆6,176Updated last week
- Postgres-native columnar storage extension☆2,901Updated 3 weeks ago
- Real-time Data Integration and Transformation: use SQL to transform, deliver, and act on fast-changing data.☆5,892Updated this week
- Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.☆1,941Updated this week
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,043Updated 5 months ago
- A cross platform way to express data transformation, relational algebra, standardized record expression and plans.☆1,264Updated 2 weeks ago
- Embeddable property graph database management system built for query speed and scalability. Implements Cypher.☆1,742Updated this week
- Stream processing platform for developers.☆7,452Updated this week
- Python Stream Processing☆1,656Updated last week
- Transmute-free Rust library to work with the Arrow format☆1,060Updated last year
- Malloy is an experimental language for describing data relationships and transformations.☆2,080Updated this week
- An extensible, state-of-the-art columnar file format☆1,118Updated this week
- 🦆 A curated list of awesome DuckDB resources☆1,559Updated this week
- GlareDB: An analytics DBMS for distributed data☆772Updated this week
- The Feldera Incremental Computation Engine☆1,172Updated this week
- Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C…☆2,556Updated 3 weeks ago
- A composable and fully extensible C++ execution engine library for data management systems.☆3,632Updated this week
- 𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://data…☆8,215Updated this week
- Apache Iceberg☆830Updated this week