Alternatives and similar repositories for open-gpu-kernel-modules

Users that are interested in open-gpu-kernel-modules are comparing it to the libraries listed below

• HazyResearch / ThunderKittens
  Tile primitives for speedy kernels
  ☆2,420Updated this week
• tspeterkim / flash-attention-minimal
  Flash Attention in ~100 lines of CUDA (forward pass only)
  ☆833Updated 5 months ago
• tinygrad / 7900xtx
  ☆445Updated last month
• mikex86 / LibreCuda
  ☆1,040Updated 2 weeks ago
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,196Updated last week
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆818Updated this week
• zeux / calm
  CUDA/Metal accelerated language model inference
  ☆581Updated last week
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆519Updated this week
• philipturner / metal-flash-attention
  FlashAttention (Metal Port)
  ☆492Updated 8 months ago
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆3,088Updated this week
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,435Updated last week
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆831Updated 9 months ago
• punica-ai / punica
  Serving multiple LoRA finetuned LLM as one
  ☆1,062Updated last year
• Cornell-RelaxML / quip-sharp
  ☆536Updated 7 months ago
• aikitoria / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆25Updated 2 weeks ago
• turboderp-org / exllamav3
  An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
  ☆375Updated last week
• mirage-project / mirage
  Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA
  ☆850Updated this week
• BobMcDear / attorch
  A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.
  ☆544Updated last week
• likejazz / llama3.cuda
  llama3.cuda is a pure C/CUDA implementation for Llama 3 model.
  ☆331Updated last month
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆639Updated last month
• NousResearch / DisTrO
  Distributed Training Over-The-Internet
  ☆932Updated 3 weeks ago
• b4rtaz / distributed-llama
  Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
  ☆2,074Updated last month
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,878Updated last year
• EGjoni / DRUGS
  Stop messing around with finicky sampling parameters and just use DRµGS!
  ☆349Updated last year
• distantmagic / paddler
  Stateful load balancer custom-tailored for llama.cpp 🏓🦙
  ☆764Updated last week
• NVIDIA / TransformerEngine
  A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada and Bla…
  ☆2,450Updated this week
• Infini-AI-Lab / Sequoia
  scalable and robust tree-based speculative decoding algorithm
  ☆345Updated 4 months ago
• microsoft / BitBLAS
  BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.
  ☆619Updated last month
• ROCm / aiter
  AI Tensor Engine for ROCm
  ☆201Updated this week
• casper-hansen / AutoAWQ
  AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
  ☆2,176Updated 3 weeks ago