Distributed inference for mobile, desktop and server.
☆3,133Apr 24, 2026Updated 6 months ago
Alternatives and similar repositories for cake
Users that are interested in cake are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Fast, flexible LLM inference☆7,815Updated this week
- Minimalist ML framework for Rust☆21,279Updated this week
- Run frontier AI locally.☆48,214Updated this week
- Claude Engineer is an interactive command-line interface (CLI) that leverages the power of Anthropic's Claude-3.5-Sonnet model to assist …☆11,215Dec 12, 2024Updated last year
- Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.☆3,131Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.☆16,245Updated this week
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆740Oct 18, 2026Updated 2 weeks ago
- [Unmaintained, see README] An ecosystem of Rust libraries for working with large language models☆6,145Jun 24, 2024Updated 2 years ago
- Universal LLM Deployment Engine with ML Compilation☆23,308Oct 30, 2026Updated last week
- A library for building fast, reliable and evolvable network services.☆27,808Updated this week
- Distribute and run LLMs with a single file.☆26,576Updated this week
- YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automatin…☆22,250Updated this week
- 🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)☆6,942Jul 4, 2025Updated last year
- High-speed Large Language Model Serving for Local Deployment☆9,870May 11, 2026Updated 5 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- LLM inference in C/C++☆133,556Updated this week
- Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cl…☆35,446Updated this week
- ⚙️🦀 Build modular and scalable LLM Applications in Rust☆9,079Updated this week
- The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.☆68,433Updated this week
- Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust☆16,401Updated this week
- A lightweight library for portable low-level GPU computation using WebGPU.☆3,994Aug 21, 2026Updated 2 months ago
- The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge☆1,663Feb 8, 2026Updated 8 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆96,465Updated this week
- Self-hosted AI coding assistant☆33,918Jun 30, 2026Updated 4 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- See, hear, and interact on your local devices.☆27,081Oct 30, 2026Updated last week
- Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks☆7,933Dec 12, 2025Updated 10 months ago
- Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audi…☆11,443Sep 9, 2026Updated last month
- Apache OpenDAL: One Layer, All Storage.☆5,456Updated this week
- 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading☆10,696Sep 7, 2024Updated 2 years ago
- Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.☆79,063Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,864Jul 14, 2026Updated 3 months ago
- `llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tas…☆1,602Oct 31, 2024Updated 2 years ago
- High-performance In-browser LLM Inference Engine☆19,356Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Inference Llama 2 in one file of pure C☆20,252Aug 6, 2024Updated 2 years ago
- A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations☆19,661Oct 28, 2026Updated last week
- Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models☆2,994Jan 7, 2025Updated last year
- MLX: An array framework for Apple silicon☆29,074Updated this week
- LLM training in simple, raw C/CUDA☆31,401Jun 26, 2025Updated last year
- A blazing fast inference solution for text embeddings models☆5,109Updated this week
- a fast cross platform AI inference engine 🤖 using Rust 🦀 and WebGPU 🎮☆468Jan 4, 2025Updated last year