Alternatives and similar repositories for dashboard

Users that are interested in dashboard are comparing it to the libraries listed below

• InternLM / turbomind
  ☆86Updated 3 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆131Updated 4 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 6 months ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆76Updated this week
• neuralmagic / AutoFP8
  ☆194Updated last month
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆129Updated this week
• deepspeedai / DeepSpeed-Kernels
  ☆72Updated 2 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆79Updated 9 months ago
• LLM-inference-router / vllm-router
  vLLM Router
  ☆29Updated last year
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆100Updated last year
• snowflakedb / ArcticInference
  ☆159Updated this week
• microsoft / chunk-attention
  ☆77Updated 2 months ago
• flashinfer-ai / cutlass-viz
  ☆60Updated 2 months ago
• Dao-AILab / grouped-latent-attention
  ☆114Updated 3 weeks ago
• Bruce-Lee-LY / decoding_attention
  Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.
  ☆38Updated 2 weeks ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆78Updated 4 months ago
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆102Updated 2 months ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆199Updated 4 months ago
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆128Updated 2 months ago
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆70Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆76Updated 2 weeks ago
• tile-ai / AttentionEngine
  ☆49Updated last month
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆151Updated this week
• AlibabaPAI / FLASHNN
  ☆96Updated 9 months ago
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated last month
• casper-hansen / AutoAWQ_kernels
  ☆74Updated 6 months ago
• ROCm / TransformerEngine
  ☆38Updated this week
• bentoml / llm-bench
  ☆54Updated 7 months ago
• Equationliu / Kangaroo
  [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…
  ☆57Updated last year
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆172Updated 2 months ago