Alternatives and similar repositories for distributed-llama:

Users that are interested in distributed-llama are comparing it to the libraries listed below

• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆3,944Updated this week
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,288Updated this week
• distantmagic / paddler
  Stateful load balancer custom-tailored for llama.cpp 🏓🦙
  ☆704Updated 3 weeks ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆511Updated this week
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆2,355Updated this week
• ggml-org / llama.vscode
  VS Code extension for LLM-assisted code/text completion
  ☆509Updated this week
• theroyallab / tabbyAPI
  An OAI compatible exllamav2 API that's both lightweight and fast
  ☆778Updated this week
• MeetKai / functionary
  Chat language model that can use tools and interpret the results
  ☆1,513Updated this week
• EricLBuehler / mistral.rs
  Blazingly fast LLM inference.
  ☆4,977Updated this week
• RahulSChand / gpu_poor
  Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
  ☆1,220Updated 2 months ago
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,160Updated 4 months ago
• NousResearch / DisTrO
  Distributed Training Over-The-Internet
  ☆878Updated 2 months ago
• evilsocket / cake
  Distributed LLM and StableDiffusion inference for mobile, desktop and server.
  ☆2,763Updated 3 months ago
• McGill-NLP / webllama
  Llama-3 agents that can browse the web by following instructions and talking to you
  ☆1,387Updated 2 months ago
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆5,247Updated this week
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆527Updated 2 months ago
• tinygrad / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆1,017Updated last month
• ILikeAI / AlwaysReddy
  AlwaysReddy is a LLM voice assistant that is always just a hotkey away.
  ☆713Updated 2 weeks ago
• abgulati / LARS
  An application for running LLMs locally on your device, with your documents, facilitating detailed citations in generated responses.
  ☆537Updated 3 months ago
• RWKV / rwkv.cpp
  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
  ☆1,468Updated 3 weeks ago
• andrewnguonly / Lumos
  A RAG LLM co-pilot for browsing the web, powered by local LLMs
  ☆1,475Updated 2 weeks ago
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,822Updated last year
• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,838Updated last year
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆747Updated this week
• kevmo314 / scuda
  SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.
  ☆1,637Updated this week
• NVIDIA / RULER
  This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
  ☆918Updated 2 weeks ago
• NousResearch / Hermes-Function-Calling
  ☆801Updated 5 months ago
• FailSpy / abliterator
  Simple Python library/structure to ablate features in LLMs which are supported by TransformerLens
  ☆406Updated 8 months ago
• LlamaEdge / LlamaEdge
  The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge
  ☆1,245Updated this week
• janhq / cortex.cpp
  Local AI API Platform
  ☆2,451Updated this week