Alternatives and similar repositories for nitro

Users that are interested in nitro are comparing it to the libraries listed below

• tile-ai / tvm
  Open deep learning compiler stack for cpu, gpu and specialized accelerators
  ☆19Updated last week
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆42Updated last year
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆26Updated 2 years ago
• eth-easl / deltazip
  Compression for Foundation Models
  ☆33Updated 3 months ago
• metacarbon / shareAtt
  Beyond KV Caching: Shared Attention for Efficient LLMs
  ☆19Updated 11 months ago
• IntelLabs / Hardware-Aware-Automated-Machine-Learning
  ☆61Updated last month
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆54Updated last year
• PipeFusion / PipeFusion
  A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters
  ☆47Updated 11 months ago
• GATECH-EIC / ShiftAddLLM
  ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
  ☆109Updated 9 months ago
• deepspeedai / DeepSpeed-Kernels
  ☆74Updated 3 months ago
• wdlctc / headinfer
  ☆52Updated 2 months ago
• sgl-project / tensorrt-demo
  TensorRT LLM Benchmark Configuration
  ☆13Updated 11 months ago
• tile-ai / TileOPs
  ☆42Updated this week
• tile-ai / AttentionEngine
  ☆49Updated last month
• mobiusml / low-rank-llama2
  Low-Rank Llama Custom Training
  ☆23Updated last year
• li-plus / flash-preference
  Accelerate LLM preference tuning via prefix sharing with a single line of code
  ☆42Updated last week
• AutonomicPerfectionist / PipeInfer
  PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
  ☆30Updated 8 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆40Updated last year
• graphcore-research / unit-scaling-demo
  Unit Scaling demo and experimentation code
  ☆16Updated last year
• gau-nernst / quantized-training
  Explore training for quantized models
  ☆20Updated this week
• tonyzhang617 / nomad-dist
  ☆36Updated last year
• mit-han-lab / tinychat-tutorial
  ☆71Updated 8 months ago
• fw-ai / llama-cuda-graph-example
  Example of applying CUDA graphs to LLaMA-v2
  ☆12Updated last year
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆80Updated 10 months ago
• Qualcomm-AI-research / gptvq
  ☆32Updated last year
• IBM / triton-dejavu
  Framework to reduce autotune overhead to zero for well known deployments.
  ☆79Updated last week
• IST-DASLab / MicroAdam
  This repository contains code for the MicroAdam paper.
  ☆20Updated 7 months ago
• flashinfer-ai / cutlass-viz
  ☆60Updated 2 months ago
• microsoft / AttentionEngine
  ☆74Updated last month
• Cornell-RelaxML / qtip
  ☆139Updated 3 weeks ago