Alternatives and similar repositories for distributed-llama

Users that are interested in distributed-llama are comparing it to the libraries listed below

• janhq / cortex.cpp
  Local AI API Platform
  ☆2,761Updated 6 months ago
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,426Updated last month
• mostlygeek / llama-swap
  Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
  ☆2,209Updated last week
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,631Updated this week
• intentee / paddler
  Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙
  ☆1,425Updated last week
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆777Updated this week
• kevmo314 / scuda
  SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.
  ☆1,800Updated 3 weeks ago
• theroyallab / tabbyAPI
  The official API server for Exllama. OAI compatible, lightweight, and fast.
  ☆1,115Updated this week
• HazyResearch / minions
  Big & Small LLMs working together
  ☆1,249Updated this week
• EricLBuehler / mistral.rs
  Blazingly fast LLM inference.
  ☆6,379Updated this week
• ggml-org / llama.vscode
  VS Code extension for LLM-assisted code/text completion
  ☆1,135Updated last week
• evilsocket / cake
  Distributed LLM and StableDiffusion inference for mobile, desktop and server.
  ☆2,899Updated last year
• ngxson / wllama
  WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
  ☆981Updated last month
• trholding / llama2.c
  Llama 2 Everywhere (L2E)
  ☆1,526Updated 5 months ago
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆1,553Updated this week
• janhq / awesome-local-ai
  An awesome repository of local AI tools
  ☆1,813Updated last year
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆612Updated 11 months ago
• containers / ramalama
  RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for in…
  ☆2,532Updated last week
• av / harbor
  Effortlessly run LLM backends, APIs, frontends, and services with one command.
  ☆2,333Updated this week
• bodaay / HuggingFaceModelDownloader
  Simple go utility to download HuggingFace Models and Datasets
  ☆813Updated 2 weeks ago
• lyogavin / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆8,979Updated 4 months ago
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,906Updated 2 years ago
• mezbaul-h / june
  Local voice chatbot for engaging conversations, powered by Ollama, Hugging Face Transformers, and Coqui TTS Toolkit
  ☆783Updated last year
• lm-sys / RouteLLM
  A framework for serving and evaluating LLM routers - save LLM costs without compromising quality
  ☆4,553Updated last year
• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,876Updated last year
• alexpinel / Dot
  Text-To-Speech, RAG, and LLMs. All local!
  ☆1,895Updated last year
• XiongjieDai / GPU-Benchmarks-on-LLM-Inference
  Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?
  ☆1,867Updated last year
• RWKV / rwkv.cpp
  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
  ☆1,562Updated 10 months ago
• vince-lam / awesome-local-llms
  Compare open-source local LLM inference projects by their metrics to assess popularity and activeness.
  ☆705Updated 2 months ago
• michaelfeil / infinity
  Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali
  ☆2,629Updated last month