Alternatives and similar repositories for cake

Users that are interested in cake are comparing it to the libraries listed below

• EricLBuehler / mistral.rs
  Blazingly fast LLM inference.
  ☆5,644Updated this week
• b4rtaz / distributed-llama
  Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
  ☆2,074Updated last month
• menloresearch / cortex.cpp
  Local AI API Platform
  ☆2,715Updated 2 weeks ago
• gpustack / gpustack
  Manage GPU clusters for running AI models
  ☆2,778Updated this week
• kyutai-labs / moshi
  Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audi…
  ☆8,356Updated this week
• usefulsensors / moonshine
  Fast and accurate automatic speech recognition (ASR) for edge devices
  ☆2,718Updated 3 weeks ago
• kevmo314 / scuda
  SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.
  ☆1,724Updated last month
• vikhyat / moondream
  tiny vision language model
  ☆8,019Updated last week
• InternLM / lmdeploy
  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
  ☆6,427Updated this week
• gradio-app / fastrtc
  The python library for real-time communication
  ☆3,991Updated this week
• fixie-ai / ultravox
  A fast multimodal LLM for real-time voice
  ☆3,968Updated 3 months ago
• pipecat-ai / pipecat
  Open Source framework for voice and multimodal conversational AI
  ☆6,247Updated this week
• nilsherzig / LLocalSearch
  LLocalSearch is a completely locally running search aggregator using LLM Agents. The user can ask a question and the system will use a ch…
  ☆5,917Updated last month
• guinmoon / LLMFarm
  llama and other large language models on iOS and MacOS offline using GGML library.
  ☆1,777Updated 2 months ago
• PrimeIntellect-ai / prime
  prime is a framework for efficient, globally distributed training of AI models over the internet.
  ☆757Updated last week
• ai-dynamo / dynamo
  A Datacenter Scale Distributed Inference Serving Framework
  ☆4,136Updated this week
• qnguyen3 / chat-with-mlx
  An all-in-one LLMs Chat UI for Apple Silicon Mac using MLX Framework.
  ☆1,562Updated 8 months ago
• microsoft / T-MAC
  Low-bit LLM inference on CPU with lookup table
  ☆793Updated this week
• menloresearch / ichigo
  Local realtime voice AI
  ☆2,317Updated 3 months ago
• ModelTC / lightllm
  LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabili…
  ☆3,262Updated this week
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆3,088Updated this week
• sgl-project / sglang
  SGLang is a fast serving framework for large language models and vision language models.
  ☆14,814Updated this week
• ictnlp / LLaMA-Omni
  LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve spee…
  ☆2,928Updated 2 weeks ago
• ggml-org / ggml
  Tensor library for machine learning
  ☆12,614Updated this week
• XiongjieDai / GPU-Benchmarks-on-LLM-Inference
  Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?
  ☆1,633Updated last year
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆5,754Updated 2 weeks ago
• pytorch / torchchat
  Run PyTorch LLMs locally on servers, desktop and mobile
  ☆3,586Updated 2 weeks ago
• LlamaEdge / LlamaEdge
  The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge
  ☆1,402Updated this week
• QwenLM / Qwen-Agent
  Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.
  ☆9,200Updated this week
• SJTU-IPADS / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,213Updated 3 months ago