Alternatives and similar repositories for ggml-python:

Users that are interested in ggml-python are comparing it to the libraries listed below

• BlinkDL / nanoRWKV
  RWKV in nanoGPT style
  ☆187Updated 9 months ago
• IST-DASLab / qmoe
  Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".
  ☆271Updated last year
• ggml-org / p1
  LLM-based code completion engine
  ☆181Updated 2 months ago
• astramind-ai / BitMat
  An efficent implementation of the method proposed in "The Era of 1-bit LLMs"
  ☆154Updated 5 months ago
• fpgaminer / GPTQ-triton
  GPTQ inference Triton kernel
  ☆298Updated last year
• kroggen / mamba.c
  Inference of Mamba models in pure C
  ☆186Updated last year
• Cornell-RelaxML / quip-sharp
  ☆528Updated 4 months ago
• thomasantony / llamacpp-python
  Python bindings for llama.cpp
  ☆200Updated last year
• Cornell-RelaxML / QuIP
  Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"
  ☆362Updated last year
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆196Updated 8 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆262Updated 5 months ago
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆770Updated this week
• NolanoOrg / cformers
  SoTA Transformers with C-backend for fast inference on your CPU.
  ☆311Updated last year
• harrisonvanderbyl / rwkv-cpp-accelerated
  A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependenci…
  ☆310Updated last year
• apple / ml-recurrent-drafter
  ☆203Updated 2 months ago
• cognitivecomputations / laserRMT
  This is our own implementation of 'Layer Selective Rank Reduction'
  ☆233Updated 9 months ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆103Updated 2 months ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆193Updated last month
• thomasgauthier / LoRD
  Low-Rank adapter extraction for fine-tuned transformers models
  ☆171Updated 10 months ago
• intel / auto-round
  Advanced Quantization Algorithm for LLMs/VLMs.
  ☆394Updated this week
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆267Updated this week
• rafacelente / bllama
  1.58-bit LLaMa model
  ☆82Updated 11 months ago
• eugenepentland / landmark-attention-qlora
  Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA
  ☆123Updated last year
• euclaise / supertrainer2000
  ☆49Updated last year
• philipturner / metal-flash-attention
  FlashAttention (Metal Port)
  ☆457Updated 6 months ago
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆93Updated last year
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆217Updated this week
• pranavjad / tinyllama-bitnet
  Train your own small bitnet model
  ☆65Updated 5 months ago
• dust-tt / llama-ssp
  Experiments on speculative sampling with Llama models
  ☆125Updated last year
• staghado / vit.cpp
  Inference Vision Transformer (ViT) in plain C/C++ with ggml
  ☆262Updated 11 months ago