lancedb / lanceLinks
Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
☆5,449Updated last week
Alternatives and similar repositories for lance
Users that are interested in lance are comparing it to the libraries listed below
Sorting:
- Apache DataFusion SQL Query Engine☆7,844Updated this week
- Distributed query engine providing simple and reliable data processing for any modality and scale☆4,546Updated this week
- Distributed stream processing engine in Rust☆4,557Updated 2 weeks ago
- Developer-friendly, embedded retrieval engine for multimodal AI. Search More; Manage Less.☆7,666Updated this week
- A native Rust library for Delta Lake, with bindings into Python☆2,965Updated this week
- Apache DataFusion Ballista Distributed Query Engine☆1,851Updated last week
- Apache OpenDAL: One Layer, All Storage.☆4,455Updated last week
- Official Rust implementation of Apache Arrow☆3,166Updated this week
- Fastest library to load data from DB to DataFrames in Rust and Python☆2,418Updated last month
- 𝗔𝗜-𝗡𝗮𝘁𝗶𝘃𝗲 𝗗𝗮𝘁𝗮 𝗪𝗮𝗿𝗲𝗵𝗼𝘂𝘀𝗲. Open-source Snowflake alternative. Proven at petabyte scale with enterprise performance. B…☆8,892Updated this week
- Real-time event streaming platform. Streaming CDC, stream processing, low-latency serving, and Iceberg management.☆8,409Updated this week
- An extensible, state of the art columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Li…☆1,791Updated this week
- A composable and fully extensible C++ execution engine library for data management systems.☆3,906Updated this week
- Making data lake work for time series☆1,183Updated last year
- Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.☆2,108Updated 7 months ago
- Python Stream Processing☆1,853Updated 6 months ago
- the portable Python dataframe library☆6,136Updated this week
- Real-time Data Integration and Transformation: use SQL to transform, deliver, and act on fast-changing data.☆6,128Updated this week
- Open-source, cloud-native, unified observability database for metrics, logs and traces, supporting SQL/PromQL/Streaming. Available on Gre…☆5,565Updated this week
- GlareDB: A light and fast SQL database for analytics☆974Updated last week
- DuckDB-powered Postgres for high performance apps & analytics.☆2,593Updated this week
- Apache Iceberg☆1,098Updated last week
- A cross platform way to express data transformation, relational algebra, standardized record expression and plans.☆1,391Updated this week
- chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse☆2,489Updated this week
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,117Updated 6 months ago
- Postgres-native columnar storage extension☆2,990Updated 7 months ago
- Postgres with GPUs for ML/AI apps.☆6,578Updated 3 months ago
- The Auron accelerator for distributed computing framework (e.g., Spark) leverages native vectorized execution to accelerate query process…☆1,609Updated this week
- Extensible SQL Lexer and Parser for Rust☆3,206Updated this week
- DuckLake is an integrated data lake and catalog format☆2,101Updated this week