Alternatives and similar repositories for Torch-TRTLLM:

Users that are interested in Torch-TRTLLM are comparing it to the libraries listed below

• SqueezeBits / QUICK
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  ☆116Updated last year
• kakaobrain / trident
  A performance library for machine learning applications.
  ☆184Updated last year
• DeepAuto-AI / hip-attention
  Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.
  ☆127Updated this week
• friendliai / FAI-Model
  FriendliAI Model Hub
  ☆92Updated 2 years ago
• ai-compiler-study / triton-kernels
  Triton kernels for Flux
  ☆20Updated 3 months ago
• EleutherAI / oslo
  OSLO: Open Source for Large-scale Optimization
  ☆175Updated last year
• pytorch-tpu / llama
  Inference code for LLaMA models
  ☆21Updated 3 weeks ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆197Updated 2 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆72Updated 7 months ago
• huggingface / kernels
  Load compute kernels from the Hub
  ☆115Updated this week
• radha-patel / SySTeC
  Performant kernels for symmetric tensors
  ☆13Updated 8 months ago
• SqueezeBits / owlite
  OwLite is a low-code AI model compression toolkit for AI models.
  ☆43Updated 2 months ago
• junstar92 / nvidia-libraries-study
  ☆52Updated 5 months ago
• MekkCyber / CutlassAcademy
  A curated collection of resources, tutorials, and best practices for learning and mastering NVIDIA CUTLASS
  ☆165Updated last month
• UmerHA / triton_util
  Make triton easier
  ☆47Updated 10 months ago
• swsnu / aisys2023
  ☆101Updated last year
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆291Updated this week
• pytorch-labs / superblock
  A block oriented training approach for inference time optimization.
  ☆32Updated 8 months ago
• pytorch-labs / tritonbench
  Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.
  ☆116Updated this week
• mgmalek / efficient_cross_entropy
  ☆101Updated 10 months ago
• daemyung / practice-triton
  삼각형의 실전! Triton
  ☆16Updated last year
• Adlik / smoothquantplus
  [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  ☆22Updated last year
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆40Updated last year
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆115Updated 4 months ago
• deepspeedai / DeepSpeed-Kernels
  ☆68Updated last month
• vedantroy / gpu_kernels
  ☆26Updated last year
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆370Updated this week
• gpu-mode / ring-attention
  ring-attention experiments
  ☆130Updated 6 months ago
• jundaf2 / INT8-Flash-Attention-FMHA-Quantization
  ☆157Updated last year
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆262Updated last month