Alternatives and similar repositories for distributed-llama

Users that are interested in distributed-llama are comparing it to the libraries listed below

• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,379Updated 3 months ago
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,603Updated 3 weeks ago
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆1,387Updated this week
• intentee / paddler
  Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙
  ☆1,388Updated last week
• evilsocket / cake
  Distributed LLM and StableDiffusion inference for mobile, desktop and server.
  ☆2,899Updated last year
• XiongjieDai / GPU-Benchmarks-on-LLM-Inference
  Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?
  ☆1,848Updated last year
• mostlygeek / llama-swap
  Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
  ☆2,025Updated last week
• janhq / cortex.cpp
  Local AI API Platform
  ☆2,764Updated 5 months ago
• vitoplantamura / OnnxStream
  Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but…
  ☆2,011Updated 3 weeks ago
• tinygrad / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆1,294Updated 6 months ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆753Updated this week
• EricLBuehler / mistral.rs
  Blazingly fast LLM inference.
  ☆6,262Updated last week
• kevmo314 / scuda
  SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.
  ☆1,785Updated 5 months ago
• av / harbor
  Effortlessly run LLM backends, APIs, frontends, and services with one command.
  ☆2,183Updated this week
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,904Updated 2 years ago
• abgulati / LARS
  An application for running LLMs locally on your device, with your documents, facilitating detailed citations in generated responses.
  ☆620Updated last year
• ggml-org / llama.vscode
  VS Code extension for LLM-assisted code/text completion
  ☆1,082Updated 3 weeks ago
• janhq / ichigo
  Local realtime voice AI
  ☆2,386Updated 2 weeks ago
• theroyallab / tabbyAPI
  The official API server for Exllama. OAI compatible, lightweight, and fast.
  ☆1,097Updated this week
• containers / ramalama
  RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for in…
  ☆2,372Updated this week
• ngxson / wllama
  WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
  ☆946Updated 2 weeks ago
• leejet / stable-diffusion.cpp
  Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++
  ☆4,809Updated this week
• alexpinel / Dot
  Text-To-Speech, RAG, and LLMs. All local!
  ☆1,844Updated last year
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆610Updated 9 months ago
• trholding / llama2.c
  Llama 2 Everywhere (L2E)
  ☆1,521Updated 3 months ago
• mistralai / mistral-finetune
  ☆3,040Updated 3 weeks ago
• ictnlp / LLaMA-Omni
  LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve spee…
  ☆3,103Updated 6 months ago
• mikex86 / LibreCuda
  ☆1,070Updated 6 months ago
• SJTU-IPADS / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,460Updated 4 months ago
• huggingface / smollm
  Everything about the SmolLM and SmolVLM family of models
  ☆3,445Updated 3 weeks ago