tonyzhang617 / nomad-dist

Related projects ⓘ

Alternatives and complementary repositories for nomad-dist

• hao-ai-lab / MuxServe
  ☆46Updated 5 months ago
• abhibambhaniya / GenZ-LLM-Analyzer
  LLM Inference analyzer for different hardware platforms
  ☆42Updated this week
• chhzh123 / ptc-tutorial
  PyTorch compilation tutorial covering TorchScript, torch.fx, and Slapo
  ☆19Updated last year
• pku-liang / MAGIS
  MAGIS: Memory Optimization via Coordinated Graph Transformation and Scheduling for DNN (ASPLOS'24)
  ☆44Updated 5 months ago
• pku-liang / ArkVale
  ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)
  ☆17Updated last week
• LeiWang1999 / Stream-k.tvm
  ☆18Updated last month
• WukLab / preble
  Stateful LLM Serving
  ☆38Updated 3 months ago
• heheda12345 / MagPy
  ☆33Updated 5 months ago
• YaoJiayi / CacheBlend
  ☆51Updated last month
• AlibabaResearch / flash-llm
  Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
  ☆181Updated last year
• VITA-Group / Q-Hitter
  ☆9Updated 5 months ago
• ucamrl / xrlflow
  ☆13Updated last year
• tlc-pack / libflash_attn
  Standalone Flash Attention v2 kernel without libtorch dependency
  ☆98Updated 2 months ago
• SNU-ARC / flashneuron
  ☆31Updated last year
• hgyhungry / alcop-artifact
  ☆21Updated last year
• ankan-ban / llama_cu_awq
  llama INT4 cuda inference with AWQ
  ☆48Updated 4 months ago
• TiledTensor / ThrillerFlow
  ThrillerFlow is a Dataflow Analysis and Codegen Framework written in Rust.
  ☆11Updated last month
• triton-lang / kernels
  ☆45Updated 2 weeks ago
• DerrickYLJ / TidalDecode
  TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆21Updated last month
• shadowpa0327 / Palu
  Code for Palu: Compressing KV-Cache with Low-Rank Projection
  ☆57Updated this week
• apuaaChen / EVT_AE
  Artifacts of EVT ASPLOS'24
  ☆17Updated 8 months ago
• tlc-pack / cutlass_fpA_intB_gemm
  A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer
  ☆85Updated 8 months ago
• jiazhihao / attention_superoptimizer
  An Attention Superoptimizer
  ☆20Updated 6 months ago
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆82Updated 2 months ago
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆53Updated 3 weeks ago
• Shigangli / Magicube
  Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.
  ☆81Updated last year
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆147Updated 4 months ago
• YukeWang96 / MGG_OSDI23
  Artifact for OSDI'23: MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Mult…
  ☆37Updated 8 months ago
• TorchMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆103Updated 3 months ago
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆114Updated 2 months ago