abetlen / llama-cpp-pythonLinks
Python bindings for llama.cpp
☆9,546Updated 3 weeks ago
Alternatives and similar repositories for llama-cpp-python
Users that are interested in llama-cpp-python are comparing it to the libraries listed below
Sorting:
- Tensor library for machine learning☆13,134Updated this week
- A fast inference library for running LLMs locally on modern consumer-class GPUs☆4,309Updated 3 weeks ago
- Large Language Model Text Generation Inference☆10,491Updated this week
- LLM inference in C/C++☆86,121Updated last week
- An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.☆4,942Updated 5 months ago
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆2,898Updated last year
- Go ahead and axolotl questions☆10,365Updated this week
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,876Updated last year
- High-speed Large Language Model Serving for Local Deployment☆8,328Updated last month
- LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath☆9,458Updated 3 months ago
- QLoRA: Efficient Finetuning of Quantized LLMs☆10,656Updated last year
- Tools for merging pretrained large language models.☆6,258Updated 3 weeks ago
- Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.☆11,758Updated this week
- Accessible large language models via k-bit quantization for PyTorch.☆7,567Updated this week
- OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset☆7,526Updated 2 years ago
- OpenChat: Advancing Open-source Language Models with Imperfect Data☆5,430Updated last year
- The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.☆8,741Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆57,435Updated this week
- Fast inference engine for Transformer models☆4,012Updated 5 months ago
- Universal LLM Deployment Engine with ML Compilation☆21,302Updated this week
- 🤗 AutoTrain Advanced☆4,484Updated 7 months ago
- Structured Outputs☆12,522Updated last week
- 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading☆9,791Updated last year
- The definitive Web UI for local AI, with powerful features and easy setup.☆44,900Updated last week
- Instruct-tune LLaMA on consumer hardware☆18,957Updated last year
- Inference Llama 2 in one file of pure C☆18,735Updated last year
- Official inference library for Mistral models☆10,452Updated 5 months ago
- 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.☆19,498Updated last week
- Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)☆12,396Updated 2 weeks ago
- TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizati…☆11,531Updated this week