Alternatives and similar repositories for llama-cpp-python-exploit

Users that are interested in llama-cpp-python-exploit are comparing it to the libraries listed below

• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,879Updated 2 years ago
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,174Updated last year
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,908Updated 2 years ago
• Cornell-RelaxML / quip-sharp
  ☆577Updated last year
• dropbox / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆912Updated last month
• Tiiny-AI / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,635Updated 2 weeks ago
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,431Updated last month
• tinygrad / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆1,316Updated 8 months ago
• dvmazur / mixtral-offloading
  Run Mixtral-8x7B models in Colab or consumer desktops
  ☆2,325Updated last year
• srush / Triton-Puzzles
  Puzzles for learning Triton
  ☆2,283Updated last year
• tairov / llama2.mojo
  Inference Llama 2 in one file of pure 🔥
  ☆2,116Updated 2 months ago
• abacaj / fine-tune-mistral
  Fine-tune mistral-7B on 3090s, a100s, h100s
  ☆724Updated 2 years ago
• RWKV / rwkv.cpp
  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
  ☆1,563Updated 10 months ago
• AnswerDotAI / fsdp_qlora
  Training LLMs with QLoRA + FSDP
  ☆1,537Updated last year
• HazyResearch / ThunderKittens
  Tile primitives for speedy kernels
  ☆3,120Updated this week
• mit-han-lab / TinyChatEngine
  TinyChatEngine: On-Device LLM Inference Library
  ☆941Updated last year
• tysam-code / hlb-gpt
  Minimalistic, extremely fast, and hackable researcher's toolbench for GPT models in 307 lines of code. Reaches <3.8 validation loss on wi…
  ☆355Updated last year
• hidet-org / hidet
  An open-source efficient deep learning framework/compiler, written in python.
  ☆740Updated 5 months ago
• SqueezeAILab / SqueezeLLM
  [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization
  ☆713Updated last year
• Vahe1994 / SpQR
  ☆553Updated last year
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,641Updated 2 weeks ago
• abetlen / llama-cpp-python
  Python bindings for llama.cpp
  ☆9,958Updated 5 months ago
• neuralmagic / deepsparse
  Sparsity-aware deep learning inference runtime for CPUs
  ☆3,161Updated 8 months ago
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆1,005Updated last year
• Lightning-AI / lightning-thunder
  PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily wri…
  ☆1,439Updated this week
• jzhang38 / TinyLlama
  The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
  ☆8,886Updated last year
• IST-DASLab / gptq
  Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
  ☆2,249Updated last year
• qwopqwop200 / GPTQ-for-LLaMa
  4 bits quantization of LLaMA using GPTQ
  ☆3,075Updated last year
• philipturner / metal-flash-attention
  FlashAttention (Metal Port)
  ☆579Updated last year
• jondurbin / airoboros
  Customizable implementation of the self-instruct paper.
  ☆1,049Updated last year