Alternatives and similar repositories for llama.cpp:

Users that are interested in llama.cpp are comparing it to the libraries listed below

• abetlen / llama-cpp-python
  Python bindings for llama.cpp
  ☆8,264Updated this week
• ggerganov / ggml
  Tensor library for machine learning
  ☆11,337Updated this week
• oobabooga / text-generation-webui
  A Gradio web UI for Large Language Models.
  ☆41,032Updated this week
• meta-llama / llama
  Inference code for Llama models
  ☆56,731Updated 3 months ago
• vllm-project / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆31,618Updated this week
• lm-sys / FastChat
  An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
  ☆37,203Updated this week
• ggerganov / whisper.cpp
  Port of OpenAI's Whisper model in C/C++
  ☆36,103Updated this week
• tloen / alpaca-lora
  Instruct-tune LLaMA on consumer hardware
  ☆18,696Updated 4 months ago
• tatsu-lab / stanford_alpaca
  Code and documentation to train Stanford's Alpaca models, and generate the data.
  ☆29,628Updated 4 months ago
• nomic-ai / gpt4all
  GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
  ☆71,070Updated this week
• cocktailpeanut / dalai
  The simplest way to run LLaMA on your local machine
  ☆13,098Updated 5 months ago
• ollama / ollama
  Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.
  ☆101,577Updated this week
• run-llama / llama_index
  LlamaIndex is a data framework for your LLM applications
  ☆37,200Updated this week
• mlc-ai / mlc-llm
  Universal LLM Deployment Engine with ML Compilation
  ☆19,346Updated this week
• haotian-liu / LLaVA
  [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
  ☆20,634Updated 4 months ago
• artidoro / qlora
  QLoRA: Efficient Finetuning of Quantized LLMs
  ☆10,096Updated 6 months ago
• karpathy / llama2.c
  Inference Llama 2 in one file of pure C
  ☆17,542Updated 4 months ago
• meta-llama / codellama
  Inference code for CodeLlama models
  ☆16,086Updated 4 months ago
• LAION-AI / Open-Assistant
  OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamical…
  ☆37,109Updated 3 months ago
• huggingface / text-generation-inference
  Large Language Model Text Generation Inference
  ☆9,206Updated this week
• antimatter15 / alpaca.cpp
  Locally run an Instruction-Tuned Chat-Style LLM
  ☆10,251Updated last year
• nlpxucan / WizardLM
  LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
  ☆9,287Updated 4 months ago
• unslothai / unsloth
  Finetune Llama 3.3, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
  ☆19,035Updated this week
• jzhang38 / TinyLlama
  The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
  ☆7,993Updated 7 months ago
• bigscience-workshop / petals
  🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
  ☆9,276Updated 3 months ago
• meta-llama / llama-recipes
  Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom dataset…
  ☆15,516Updated this week
• mudler / LocalAI
  The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on…
  ☆26,888Updated this week
• mistralai / mistral-inference
  Official inference library for Mistral models
  ☆9,784Updated last month
• karpathy / nanoGPT
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆37,791Updated this week
• Lightning-AI / litgpt
  20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
  ☆10,892Updated this week