Alternatives and similar repositories for distributed-llama

Users that are interested in distributed-llama are comparing it to the libraries listed below

• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,477Updated this week
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,236Updated this week
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆661Updated this week
• distantmagic / paddler
  Stateful load balancer custom-tailored for llama.cpp 🏓🦙
  ☆792Updated this week
• menloresearch / cortex.cpp
  Local AI API Platform
  ☆2,765Updated 2 weeks ago
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆686Updated this week
• XiongjieDai / GPU-Benchmarks-on-LLM-Inference
  Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?
  ☆1,702Updated last year
• mostlygeek / llama-swap
  Model swapping for llama.cpp (or any local OpenAPI compatible server)
  ☆1,048Updated this week
• tinygrad / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆1,188Updated last month
• EricLBuehler / mistral.rs
  Blazingly fast LLM inference.
  ☆5,890Updated this week
• evilsocket / cake
  Distributed LLM and StableDiffusion inference for mobile, desktop and server.
  ☆2,876Updated 8 months ago
• ggml-org / llama.vscode
  VS Code extension for LLM-assisted code/text completion
  ☆842Updated 2 weeks ago
• theroyallab / tabbyAPI
  The official API server for Exllama. OAI compatible, lightweight, and fast.
  ☆1,000Updated last week
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,887Updated last year
• ex3ndr / llama-coder
  Replace Copilot local AI
  ☆2,029Updated last year
• NousResearch / DisTrO
  Distributed Training Over-The-Internet
  ☆946Updated 2 months ago
• ngxson / wllama
  WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
  ☆770Updated this week
• trholding / llama2.c
  Llama 2 Everywhere (L2E)
  ☆1,519Updated 6 months ago
• kevmo314 / scuda
  SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.
  ☆1,744Updated last month
• Lizonghang / prima.cpp
  prima.cpp: Speeding up 70B-scale LLM inference on low-resource everyday home clusters
  ☆975Updated this week
• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,868Updated last year
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,169Updated 9 months ago
• ParisNeo / ollama_proxy_server
  A proxy server for multiple ollama instances with Key security
  ☆462Updated last week
• ILikeAI / AlwaysReddy
  AlwaysReddy is a LLM voice assistant that is always just a hotkey away.
  ☆743Updated 4 months ago
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆578Updated 5 months ago
• RahulSChand / gpu_poor
  Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
  ☆1,329Updated 7 months ago
• abgulati / LARS
  An application for running LLMs locally on your device, with your documents, facilitating detailed citations in generated responses.
  ☆602Updated 8 months ago
• RWKV / rwkv.cpp
  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
  ☆1,532Updated 3 months ago
• Blaizzy / mlx-vlm
  MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
  ☆1,498Updated this week
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆3,309Updated last month