Alternatives and similar repositories for bitsandbytes:

Users that are interested in bitsandbytes are comparing it to the libraries listed below

• ROCm / flash-attention
  Fast and memory-efficient exact attention
  ☆152Updated this week
• ROCm / AITemplate
  AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (N…
  ☆11Updated 7 months ago
• ROCm / triton
  Development repository for the Triton language and compiler
  ☆104Updated this week
• HanGuo97 / flute
  Fast Matrix Multiplications for Lookup Table-Quantized LLMs
  ☆221Updated last week
• Cornell-RelaxML / qtip
  ☆100Updated last month
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆66Updated this week
• agrocylo / bitsandbytes-rocm
  8-bit CUDA functions for PyTorch, ported to HIP for use in AMD GPUs
  ☆46Updated last year
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆64Updated 2 months ago
• ROCm / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆57Updated this week
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆199Updated last week
• usyd-fsalab / fp6_llm
  An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).
  ☆232Updated 3 months ago
• cchan / tccl
  extensible collectives library in triton
  ☆77Updated 4 months ago
• Repeerc / flash-attention-v2-RDNA3-minimal
  a simple Flash Attention v2 implementation with ROCM (RDNA3 GPU, roc wmma), mainly used for stable diffusion(ComfyUI) in Windows ZLUDA en…
  ☆35Updated 5 months ago
• jundaf2 / INT8-Flash-Attention-FMHA-Quantization
  ☆157Updated last year
• ROCm / xformers
  Hackable and optimized Transformers building blocks, supporting a composable construction.
  ☆22Updated last week
• olealgoritme / gddr6
  Linux​ based GDDR6/GDDR6X VRAM temperature reader for NVIDIA RTX 3000/4000 series GPUs.
  ☆89Updated 5 months ago
• arlo-phoenix / bitsandbytes-rocm-5.6
  8-bit CUDA functions for PyTorch Rocm compatible
  ☆39Updated 10 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆64Updated 4 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆43Updated 6 months ago
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆216Updated last week
• pytorch-labs / float8_experimental
  This repository contains the experimental PyTorch native float8 training UX
  ☆219Updated 5 months ago
• Dao-AILab / fast-hadamard-transform
  Fast Hadamard transform in CUDA, with a PyTorch interface
  ☆135Updated 8 months ago
• ROCm / hipBLASLt
  hipBLASLt is a library that provides general matrix-matrix operations with a flexible API and extends functionalities beyond a traditiona…
  ☆73Updated this week
• triton-lang / kernels
  ☆64Updated 2 months ago
• wangsiping97 / FastGEMV
  High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.
  ☆94Updated 6 months ago
• zinccat / Awesome-Triton-Kernels
  Collection of kernels written in Triton language
  ☆91Updated 3 months ago
• spcl / QuaRot
  Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.
  ☆322Updated 2 months ago
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆180Updated 2 months ago
• Cornell-RelaxML / QuIP
  Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"
  ☆358Updated 11 months ago