Alternatives and similar repositories for llama.cpp

Users that are interested in llama.cpp are comparing it to the libraries listed below

• turboderp-org / exui
  Web UI for ExLlamaV2
  ☆513Updated last year
• theroyallab / tabbyAPI
  The official API server for Exllama. OAI compatible, lightweight, and fast.
  ☆1,121Updated 2 weeks ago
• oobabooga / text-generation-webui-extensions
  ☆674Updated 3 weeks ago
• wawawario2 / long_term_memory
  A gradio web UI for running Large Language Models like GPT-J 6B, OPT, GALACTICA, LLaMA, and Pygmalion.
  ☆310Updated 2 years ago
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,908Updated 2 years ago
• itsme2417 / PolyMind
  A multimodal, function calling powered LLM webui.
  ☆216Updated last year
• AndrewVeee / nucleo-ai
  An AI assistant beyond the chat box.
  ☆329Updated last year
• Maximilian-Winter / llama-cpp-agent
  The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …
  ☆615Updated 11 months ago
• keldenl / gpt-llama.cpp
  A llama.cpp drop-in replacement for OpenAI's GPT endpoints, allowing GPT-powered apps to run off local llama.cpp models instead of OpenAI…
  ☆597Updated 2 years ago
• RWKV / rwkv.cpp
  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
  ☆1,562Updated 10 months ago
• cmp-nct / ggllm.cpp
  Falcon LLM ggml framework with CPU and GPU support
  ☆249Updated 2 years ago
• turboderp-org / exllamav3
  An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
  ☆626Updated last week
• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,879Updated 2 years ago
• johnsmith0031 / alpaca_lora_4bit
  ☆535Updated 2 years ago
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,647Updated 2 weeks ago
• bigcode-project / starcoder.cpp
  C++ implementation for 💫StarCoder
  ☆459Updated 2 years ago
• flurb18 / AgentOoba
  An autonomous AI agent extension for Oobabooga's web ui
  ☆173Updated 2 years ago
• the-crypt-keeper / can-ai-code
  Self-evaluating interview for AI coders
  ☆600Updated 7 months ago
• epolewski / EricLLM
  A fast batching API to serve LLM models
  ☆189Updated last year
• bodaay / HuggingFaceModelDownloader
  Simple go utility to download HuggingFace Models and Datasets
  ☆874Updated last week
• lmg-anon / mikupad
  LLM Frontend in a single html file
  ☆694Updated last month
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,440Updated 2 months ago
• matt-c1 / llama-3-quant-comparison
  Comparison of the output quality of quantization methods, using Llama 3, transformers, GGUF, EXL2.
  ☆165Updated last year
• PABannier / bark.cpp
  Suno AI's Bark model in C/C++ for fast text-to-speech generation
  ☆854Updated last year
• oobabooga / one-click-installers
  Simplified installers for oobabooga/text-generation-webui.
  ☆565Updated 2 years ago
• xNul / code-llama-for-vscode
  Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.
  ☆569Updated last year
• henk717 / KoboldAI
  KoboldAI is generative AI software optimized for fictional use, but capable of much more!
  ☆422Updated last year
• jondurbin / airoboros
  Customizable implementation of the self-instruct paper.
  ☆1,049Updated last year
• rmihaylov / falcontune
  Tune any FALCON in 4-bit
  ☆463Updated 2 years ago
• bublint / ue5-llama-lora
  A proof-of-concept project that showcases the potential for using small, locally trainable LLMs to create next-generation documentation t…
  ☆541Updated 2 years ago