Alternatives and similar repositories for open-gpu-kernel-modules

Users that are interested in open-gpu-kernel-modules are comparing it to the libraries listed below

• mikex86 / LibreCuda
  ☆1,042Updated last month
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,216Updated 3 weeks ago
• HazyResearch / ThunderKittens
  Tile primitives for speedy kernels
  ☆2,478Updated this week
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆846Updated 9 months ago
• mirage-project / mirage
  Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA
  ☆1,322Updated this week
• ikawrakow / ik_llama.cpp
  llama.cpp fork with additional SOTA quants and improved performance
  ☆608Updated this week
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆837Updated last week
• tspeterkim / flash-attention-minimal
  Flash Attention in ~100 lines of CUDA (forward pass only)
  ☆853Updated 5 months ago
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,457Updated last week
• Cornell-RelaxML / quip-sharp
  ☆541Updated 8 months ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆646Updated 2 months ago
• likejazz / llama3.cuda
  llama3.cuda is a pure C/CUDA implementation for Llama 3 model.
  ☆332Updated 2 months ago
• philipturner / metal-flash-attention
  FlashAttention (Metal Port)
  ☆497Updated 9 months ago
• distantmagic / paddler
  Stateful load balancer custom-tailored for llama.cpp 🏓🦙
  ☆782Updated this week
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆3,239Updated this week
• punica-ai / punica
  Serving multiple LoRA finetuned LLM as one
  ☆1,066Updated last year
• zeux / calm
  CUDA/Metal accelerated language model inference
  ☆589Updated last month
• turboderp-org / exllamav3
  An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
  ☆419Updated last week
• efeslab / Nanoflow
  A throughput-oriented high-performance serving framework for LLMs
  ☆832Updated 3 weeks ago
• IST-DASLab / gptq
  Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
  ☆2,131Updated last year
• casper-hansen / AutoAWQ
  AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
  ☆2,196Updated last month
• NVIDIA / RULER
  This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
  ☆1,159Updated this week
• NousResearch / DisTrO
  Distributed Training Over-The-Internet
  ☆940Updated last month
• mit-han-lab / llm-awq
  [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
  ☆3,101Updated 2 weeks ago
• theroyallab / tabbyAPI
  The official API server for Exllama. OAI compatible, lightweight, and fast.
  ☆990Updated this week
• NVIDIA / TransformerEngine
  A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada and Bla…
  ☆2,507Updated last week
• NVIDIA / TensorRT-Model-Optimizer
  A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. …
  ☆1,006Updated last week
• Infini-AI-Lab / Sequoia
  scalable and robust tree-based speculative decoding algorithm
  ☆348Updated 5 months ago
• microsoft / BitBLAS
  BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.
  ☆637Updated last month
• microsoft / Llama-2-Onnx
  ☆1,027Updated last year