atoma-network / atoma-inferLinks
Fast serverless LLM inference, in Rust.
☆93Updated 7 months ago
Alternatives and similar repositories for atoma-infer
Users that are interested in atoma-infer are comparing it to the libraries listed below
Sorting:
- Unofficial Rust bindings to Apple's mlx framework☆192Updated last week
- Low rank adaptation (LoRA) for Candle.☆162Updated 5 months ago
- Andrej Karpathy's Let's build GPT: from scratch video & notebook implemented in Rust + candle☆76Updated last year
- Experimental compiler for deep learning models☆67Updated 2 weeks ago
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆107Updated 7 months ago
- A single-binary, GPU-accelerated LLM server (HTTP and WebSocket API) written in Rust☆79Updated last year
- Library for doing RAG☆77Updated 3 weeks ago
- A high-performance constrained decoding engine based on context free grammar in Rust☆54Updated 4 months ago
- LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!☆109Updated 2 years ago
- A simple, CUDA or CPU powered, library for creating vector embeddings using Candle and models from Hugging Face☆41Updated last year
- A comprehensive Rust translation of the code from Sebastian Raschka's Build an LLM from Scratch book.☆250Updated last week
- Dataflow is a data processing library, primarily for machine learning.☆24Updated 2 years ago
- ☆24Updated 5 months ago
- Examples and use cases for building LLM-Powered apps with Rig☆79Updated 11 months ago
- llm_utils: Basic LLM tools, best practices, and minimal abstraction.☆47Updated 7 months ago
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆237Updated 2 months ago
- Structured outputs for LLMs☆51Updated last year
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆473Updated last week
- AI gateway and observability server written in Rust. Designed to help optimize multi-agent workflows.☆64Updated last year
- 🦀 A Pure Rust Framework For Building AGI (WIP).☆105Updated last week
- An extension library to Candle that provides PyTorch functions not currently available in Candle☆39Updated last year
- Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package☆232Updated last week
- A Fish Speech implementation in Rust, with Candle.rs☆98Updated 4 months ago
- Anthropic Rust SDK 🦀 with async support.☆66Updated 7 months ago
- Use multiple LLM backends in a single crate, simple builder-based configuration, and built-in prompt chaining & templating.☆137Updated 4 months ago
- LLM Orchestrator built in Rust☆284Updated last year
- A high-performance RAG indexing pipeline implemented in Rust using LanceDB and Candle☆23Updated last year
- High-performance framework for building interactive multi-agent workflow systems in Rust☆146Updated 3 weeks ago
- Implementing the BitNet model in Rust☆39Updated last year
- ☆19Updated last year