atoma-network / atoma-inferLinks
Fast serverless LLM inference, in Rust.
☆88Updated 5 months ago
Alternatives and similar repositories for atoma-infer
Users that are interested in atoma-infer are comparing it to the libraries listed below
Sorting:
- Unofficial Rust bindings to Apple's mlx framework☆184Updated last week
- Library for doing RAG☆75Updated 2 weeks ago
- Andrej Karpathy's Let's build GPT: from scratch video & notebook implemented in Rust + candle☆74Updated last year
- Low rank adaptation (LoRA) for Candle.☆154Updated 4 months ago
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆101Updated 5 months ago
- Experimental compiler for deep learning models☆68Updated 3 months ago
- A high-performance constrained decoding engine based on context free grammar in Rust☆55Updated 3 months ago
- A simple, CUDA or CPU powered, library for creating vector embeddings using Candle and models from Hugging Face☆37Updated last year
- LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!☆108Updated 2 years ago
- A comprehensive Rust translation of the code from Sebastian Raschka's Build an LLM from Scratch book.☆233Updated last week
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆230Updated 3 weeks ago
- Examples and use cases for building LLM-Powered apps with Rig☆70Updated 10 months ago
- llm_utils: Basic LLM tools, best practices, and minimal abstraction.☆47Updated 6 months ago
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆431Updated last week
- AI gateway and observability server written in Rust. Designed to help optimize multi-agent workflows.☆63Updated last year
- Structured outputs for LLMs☆50Updated last year
- A single-binary, GPU-accelerated LLM server (HTTP and WebSocket API) written in Rust☆80Updated last year
- 🦀 A Pure Rust Framework For Building AGI (WIP).☆96Updated 2 weeks ago
- LLM Orchestrator built in Rust☆279Updated last year
- ☆20Updated 10 months ago
- A Fish Speech implementation in Rust, with Candle.rs☆94Updated 2 months ago
- A high-performance RAG indexing pipeline implemented in Rust using LanceDB and Candle☆21Updated last year
- ☆24Updated 4 months ago
- Anthropic Rust SDK 🦀 with async support.☆65Updated 6 months ago
- High-level, optionally asynchronous Rust bindings to llama.cpp☆228Updated last year
- Implementing the BitNet model in Rust☆39Updated last year
- OpenAI compatible API for serving LLAMA-2 model☆218Updated last year
- Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package☆221Updated 2 months ago
- An extension library to Candle that provides PyTorch functions not currently available in Candle☆40Updated last year
- allms: One Rust Library to rule them aLLMs☆95Updated last week