Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime.
☆346Oct 31, 2026Updated last week
Alternatives and similar repositories for xinfer
Users that are interested in xinfer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LLM Kernel Library for Rust☆32Oct 30, 2026Updated last week
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆740Oct 18, 2026Updated 2 weeks ago
- Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2☆737Updated this week
- A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler …☆520Updated this week
- Testbed for LLM inference with cutile-rs.☆109Oct 7, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Fast, flexible LLM inference☆7,815Updated this week
- Nano vLLM v1 engine☆15Aug 6, 2025Updated last year
- BlitzScale Router - Distributed LLM Inference Router (Rust)☆17May 25, 2026Updated 5 months ago
- Fast serverless LLM inference, in Rust.☆136Sep 11, 2026Updated last month
- cuTile Rust provides a safe, tile-based kernel programming DSL for the Rust programming language. It features a safe host-side API for pa…☆1,053Oct 8, 2026Updated 3 weeks ago
- The local inference server for coding agents. Pure Rust, one binary, Apple Silicon + NVIDIA. Anthropic + OpenAI APIs; the KV cache surviv…☆22Updated this week
- Lightweight LLM inference engine inspired by nano-vllm, with radix-tree based prefix cache, tp & pp, cuda graph, openai api, async schedu…☆17Mar 29, 2026Updated 7 months ago
- High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and S…☆221Updated this week
- Nano vLLM☆13Jun 26, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆12Jan 4, 2024Updated 2 years ago
- Minimalist ML framework for Rust☆21,279Updated this week
- Fast ML inference & training for ONNX models in Rust☆2,610Updated this week
- A model-agnostic GPU runtime for shipping models as verified programs.☆163Updated this week
- ☆664Updated this week
- aha model inference library, now supports Qwen(2.5VL/3/3VL/3.5/ASR/3Embedding/3Reranker), MiniCPM(4/5), VoxCPM(0.5B/1.5/2), DeepSeek-OCR/…☆396Jun 7, 2026Updated 5 months ago
- AI agent runtime for Rust — type-safe state, multi-protocol serving, plugin extensibility.☆100Sep 4, 2026Updated 2 months ago
- The Rust library for video generation models based on Candle (HF)☆29Oct 6, 2026Updated last month
- Safe rust wrapper around CUDA toolkit☆1,240Sep 24, 2026Updated last month
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package☆341Updated this week
- Speed-of-Light SW efficiency by using ultra low-latency primitives for comms collectives☆13Apr 8, 2026Updated 6 months ago
- fused MoE kernel in Candle backend☆14Feb 3, 2026Updated 9 months ago
- Native Rust, single-binary MLX inference server for Apple Silicon. OpenAI/Anthropic-compatible LLM serving — text, vision, audio, embeddi…☆17Updated this week
- NVIDIA's CUDA platform for Rust. Host runtime crates plus Tile (cutile-rs) and SIMT (cuda-oxide) kernel programming models in idiomatic R…☆4,000Updated this week
- Nano vLLM☆16,325Apr 26, 2026Updated 6 months ago
- Lightning-fast LLM inference engine - Built with Rust (inspiration from https://github.com/GeeeekExplorer/nano-vllm)☆38Jun 24, 2025Updated last year
- Aggressive decode optimizations for Qwen3-0.6B on RTX 5090☆64Feb 25, 2026Updated 8 months ago
- Rust implementation of Qwen3-TTS speech synthesis☆150Feb 6, 2026Updated 9 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Automatically derive Python dunder methods for your Rust code☆25Sep 24, 2026Updated last month
- ☆28Updated this week
- Rust port of simplewall — a Windows Filtering Platform (WFP) firewall. GPL-3.0-or-later.☆20Oct 13, 2026Updated 3 weeks ago
- W4A16 MoE GEMM CUDA kernel. Tuned for H200, B200 & B300.☆14Oct 30, 2026Updated last week
- MXFP4-compatible 4-bit floating point types and block formats for Rust.☆12Mar 4, 2026Updated 8 months ago
- PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.☆25Jan 7, 2026Updated 10 months ago
- Humming is a high-performance, lightweight, and highly flexible JIT (Just-In-Time) compiled GEMM kernel library specifically designed for…☆350Updated this week