deepseek-ai / smallpondLinks
A lightweight data processing framework built on DuckDB and 3FS.
☆4,707Updated 3 months ago
Alternatives and similar repositories for smallpond
Users that are interested in smallpond are comparing it to the libraries listed below
Sorting:
- A high-performance distributed file system designed to address the challenges of AI training and inference workloads.☆9,039Updated last week
- A bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training.☆2,807Updated 3 months ago
- Expert Parallelism Load Balancer☆1,215Updated 3 months ago
- Analyze computation-communication overlap in V3/R1.☆1,062Updated 3 months ago
- DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling☆5,468Updated this week
- Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation☆7,835Updated last month
- DeepEP: an efficient expert-parallel communication library☆8,194Updated this week
- A Datacenter Scale Distributed Inference Serving Framework☆4,326Updated this week
- FlashMLA: Efficient MLA decoding kernels☆11,623Updated 2 months ago
- Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, v…☆4,866Updated this week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆3,459Updated this week
- Cost-efficient and pluggable Infrastructure components for GenAI inference☆3,767Updated this week
- DuckDB-powered Postgres for high performance apps & analytics.☆2,335Updated this week
- Distributed query engine providing simple and reliable data processing for any modality and scale☆3,006Updated this week
- verl: Volcano Engine Reinforcement Learning for LLMs☆9,958Updated this week
- Sky-T1: Train your own O1 preview model within $450☆3,272Updated last month
- Democratizing Reinforcement Learning for LLMs☆3,396Updated last month
- DuckLake is an integrated data lake and catalog format☆1,666Updated this week
- Developer-friendly, embedded retrieval engine for multimodal AI. Search More; Manage Less.☆6,802Updated this week
- AdalFlow: The library to build & auto-optimize LLM applications.☆3,365Updated this week
- Redis for LLMs☆1,714Updated this week
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆1,553Updated this week
- FlashInfer: Kernel Library for LLM Serving☆3,239Updated this week
- SGLang is a fast serving framework for large language models and vision language models.☆15,421Updated this week
- 🦆 A curated list of awesome DuckDB resources☆1,841Updated this week
- Open, Multi-modal Catalog for Data & AI☆2,942Updated last week
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆770Updated last month
- A PyTorch native platform for training generative AI models☆3,953Updated this week
- Everything about the SmolLM2 and SmolVLM family of models☆2,590Updated 2 months ago
- Open Source Application for Advanced LLM Engineering: interact, train, fine-tune, and evaluate large language models on your own computer…☆3,454Updated this week