Alternatives and similar repositories for cake:

Users that are interested in cake are comparing it to the libraries listed below

• EricLBuehler / mistral.rs
  Blazingly fast LLM inference.
  ☆5,369Updated this week
• b4rtaz / distributed-llama
  Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
  ☆1,993Updated last week
• bklieger-groq / g1
  g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains
  ☆4,201Updated 2 months ago
• togethercomputer / MoA
  Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models
  ☆2,711Updated 2 months ago
• microsoft / BitNet
  Official inference framework for 1-bit LLMs
  ☆12,851Updated last month
• kevmo314 / scuda
  SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.
  ☆1,682Updated 3 weeks ago
• ai-dynamo / dynamo
  A Datacenter Scale Distributed Inference Serving Framework
  ☆3,377Updated this week
• LlamaEdge / LlamaEdge
  The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge
  ☆1,316Updated this week
• lm-sys / RouteLLM
  A framework for serving and evaluating LLM routers - save LLM costs without compromising quality
  ☆3,763Updated 7 months ago
• filipstrand / mflux
  A MLX port of FLUX based on the Huggingface Diffusers implementation.
  ☆1,287Updated last week
• gluonfield / enchanted
  Enchanted is iOS and macOS app for chatting with private self hosted language models such as Llama2, Mistral or Vicuna using Ollama.
  ☆5,087Updated last week
• mediar-ai / screenpipe
  AI app store powered by 24/7 desktop history. open source | 100% local | dev friendly | 24/7 screen, mic recording
  ☆12,792Updated this week
• SJTU-IPADS / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,169Updated last month
• adithya-s-k / omniparse
  Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks
  ☆6,418Updated 4 months ago
• pytorch / torchchat
  Run PyTorch LLMs locally on servers, desktop and mobile
  ☆3,548Updated this week
• lyogavin / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆5,753Updated 4 months ago
• menloresearch / cortex.cpp
  Local AI API Platform
  ☆2,579Updated this week
• NovaSky-AI / SkyThought
  Sky-T1: Train your own O1 preview model within $450
  ☆3,149Updated this week
• Blaizzy / mlx-vlm
  MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
  ☆1,105Updated this week
• linkedin / Liger-Kernel
  Efficient Triton Kernels for LLM Training
  ☆4,743Updated this week
• InternLM / MindSearch
  🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)
  ☆6,250Updated 2 months ago
• truefoundry / cognita
  RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
  ☆3,976Updated last month
• microsoft / aici
  AICI: Prompts as (Wasm) Programs
  ☆2,008Updated 2 months ago
• apple / corenet
  CoreNet: A library for training deep neural networks
  ☆6,999Updated 5 months ago
• vikhyat / moondream
  tiny vision language model
  ☆7,701Updated this week
• n4ze3m / page-assist
  Use your locally running AI models to assist you in your web browsing
  ☆6,083Updated this week
• Kiln-AI / Kiln
  The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets.
  ☆3,264Updated this week
• Doriandarko / claude-engineer
  Claude Engineer is an interactive command-line interface (CLI) that leverages the power of Anthropic's Claude-3.5-Sonnet model to assist …
  ☆10,936Updated 3 months ago
• agiresearch / AIOS
  AIOS: AI Agent Operating System
  ☆3,992Updated last week
• NousResearch / DisTrO
  Distributed Training Over-The-Internet
  ☆893Updated 3 months ago