Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.
☆603Feb 28, 2026Updated this week
Alternatives and similar repositories for candle-vllm
Users that are interested in candle-vllm are comparing it to the libraries listed below
Sorting:
- Low rank adaptation (LoRA) for Candle.☆169Apr 18, 2025Updated 10 months ago
- Fast, flexible LLM inference☆6,653Feb 27, 2026Updated last week
- Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package☆268Feb 19, 2026Updated 2 weeks ago
- Fast serverless LLM inference, in Rust.☆110Nov 5, 2025Updated 4 months ago
- Minimalist ML framework for Rust☆19Dec 4, 2025Updated 3 months ago
- Minimalist ML framework for Rust☆19,509Updated this week
- Sampling techniques for Candle.☆19Apr 3, 2024Updated last year
- Tutorial for Porting PyTorch Transformer Models to Candle (Rust)☆341Jul 22, 2024Updated last year
- Instant, controllable, local pre-trained AI models in Rust☆2,145Updated this week
- Rust library for vector embeddings and reranking.☆780Feb 23, 2026Updated last week
- An LLM interface (chat bot) implemented in pure Rust using HuggingFace/Candle over Axum Websockets, an SQLite Database, and a Leptos (Was…☆140Oct 11, 2024Updated last year
- ☆478Updated this week
- Graph model execution API for Candle☆17Jul 27, 2025Updated 7 months ago
- Fast ML inference & training for ONNX models in Rust☆2,042Updated this week
- A collection of optimisers for use with candle☆45Dec 29, 2025Updated 2 months ago
- A comprehensive Rust translation of the code from Sebastian Raschka's Build an LLM from Scratch book.☆304Updated this week
- A cross-platform browser ML framework.☆747Nov 23, 2024Updated last year
- An extension library to Candle that provides PyTorch functions not currently available in Candle☆41Mar 15, 2024Updated last year
- [Unmaintained, see README] An ecosystem of Rust libraries for working with large language models☆6,150Jun 24, 2024Updated last year
- Model Context Protocol (MCP) implementation in Rust☆352Mar 21, 2025Updated 11 months ago
- LLM Orchestrator built in Rust☆284Mar 14, 2024Updated last year
- Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.☆14,473Updated this week
- A blazing fast inference solution for text embeddings models☆4,553Feb 25, 2026Updated last week
- ☆41Nov 18, 2024Updated last year
- ⚙️🦀 Build modular and scalable LLM Applications in Rust☆6,221Updated this week
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆244Aug 6, 2025Updated 7 months ago
- A high-performance constrained decoding engine based on context free grammar in Rust☆58May 22, 2025Updated 9 months ago
- Safe rust wrapper around CUDA toolkit☆1,066Feb 27, 2026Updated last week
- a fast cross platform AI inference engine 🤖 using Rust 🦀 and WebGPU 🎮☆463Jan 4, 2025Updated last year
- Distributed inference for mobile, desktop and server.☆2,948Updated this week
- Llama2 LLM ported to Rust burn☆280Apr 16, 2024Updated last year
- Candle Pipelines provides a simple, intuitive interface for Rust developers who want to work with Large Language Models locally, powered …☆23Jan 5, 2026Updated 2 months ago
- Rust bindings for the C++ api of PyTorch.☆5,302Jan 22, 2026Updated last month
- Cookbook to build Rust Candle models☆86Nov 30, 2023Updated 2 years ago
- Rust crate for some audio utilities☆27Mar 8, 2025Updated 11 months ago
- Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)☆3,042Jan 13, 2026Updated last month
- Deep learning at the speed of light.☆2,775Updated this week
- Run Generative AI models directly on your hardware☆42Aug 7, 2024Updated last year
- Deep learning in Rust, with shape checked tensors and neural networks☆1,896Jul 23, 2024Updated last year