99991 / pygguf

Related projects ⓘ

Alternatives and complementary repositories for pygguf

• KerfuffleV2 / gguf-tools
  Some random tools for working with the GGUF file format
  ☆20Updated 11 months ago
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆46Updated 8 months ago
• astramind-ai / BitMat
  An efficent implementation of the method proposed in "The Era of 1-bit LLMs"
  ☆154Updated last month
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.
  ☆150Updated last month
• abetlen / ggml-python
  Python bindings for ggml
  ☆132Updated 2 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆87Updated this week
• tdrussell / qlora-pipe
  A pipeline parallel training script for LLMs.
  ☆83Updated this week
• ModelCloud / GPTQModel
  Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
  ☆125Updated this week
• Cornell-RelaxML / qtip
  ☆67Updated last week
• euclaise / supertrainer2000
  ☆49Updated 8 months ago
• wangitu / Ada-Instruct
  ☆17Updated 7 months ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆25Updated last year
• chu-tianxiang / llama-cpp-torch
  llama.cpp to PyTorch Converter
  ☆26Updated 7 months ago
• Infini-AI-Lab / MagicDec
  Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆79Updated this week
• OpenGVLab / EfficientQAT
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆226Updated last month
• NolanoOrg / sparse_quant_llms
  SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia
  ☆41Updated last year
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆38Updated 10 months ago
• kroggen / mamba.c
  Inference of Mamba models in pure C
  ☆178Updated 8 months ago
• monk1337 / auto-ollama
  run ollama & gguf easily with a single command
  ☆48Updated 6 months ago
• mobiusml / gemlite
  Simple and fast low-bit matmul kernels in CUDA / Triton
  ☆145Updated this week
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆173Updated 4 months ago
• casper-hansen / AutoAWQ_kernels
  ☆52Updated this week
• AlpinDale / QuIP-for-Llama
  Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees" adapted for Llama models
  ☆36Updated last year
• nyunAI / PruneGPT
  ☆53Updated 5 months ago
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆50Updated 7 months ago
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆74Updated last month
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆93Updated this week
• GreenBitAI / low_bit_llama
  Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs
  ☆111Updated 10 months ago
• SalesforceAIResearch / GemFilter
  ☆63Updated last month
• hahnyuan / PB-LLM
  PB-LLM: Partially Binarized Large Language Models
  ☆148Updated last year