HabanaAI / Megatron-DeepSpeed

Related projects: ⓘ

• microsoft / DeepSpeed-Kernels
  ☆50Updated 3 months ago
• facebookresearch / MODel_opt
  Memory Optimizations for Deep Learning (ICML 2023)
  ☆58Updated 6 months ago
• graphcore-research / unit-scaling-demo
  Unit Scaling demo and experimentation code
  ☆16Updated 6 months ago
• stanford-futuredata / stk
  ☆83Updated 3 weeks ago
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆92Updated last year
• tridao / flash-attention-wheels
  ☆38Updated 9 months ago
• SqueezeBits / QUICK
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  ☆106Updated 6 months ago
• HabanaAI / DeepSpeed
  DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
  ☆11Updated 3 weeks ago
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆134Updated 2 months ago
• Dao-AILab / fast-hadamard-transform
  Fast Hadamard transform in CUDA, with a PyTorch interface
  ☆87Updated 3 months ago
• Raincleared-Song / sparse_gpu_operator
  GPU operators for sparse tensor operations
  ☆27Updated 6 months ago
• DachengLi1 / AMP
  (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.
  ☆33Updated last year
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆47Updated 2 weeks ago
• shadowpa0327 / Palu
  Code for Palu: Compressing KV-Cache with Low-Rank Projection
  ☆39Updated this week
• kssteven418 / BigLittleDecoder
  [NeurIPS'23] Speculative Decoding with Big Little Decoder
  ☆84Updated 7 months ago
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆34Updated this week
• AlibabaPAI / FLASHNN
  ☆67Updated last week
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆20Updated 2 weeks ago
• LeiWang1999 / AutoGPTQ.tvm
  GPTQ inference TVM kernel
  ☆35Updated 4 months ago
• usyd-fsalab / fp6_llm
  An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).
  ☆173Updated 3 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆68Updated 3 months ago
• sgl-project / tensorrt-demo
  TensorRT LLM Benchmark Configuration
  ☆10Updated last month
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆29Updated 6 months ago
• AlibabaResearch / flash-llm
  Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
  ☆166Updated 11 months ago
• vedantroy / gpu_kernels
  ☆22Updated 8 months ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆25Updated last year
• hpcaitech / Elixir
  Elixir: Train a Large Language Model on a Small GPU Cluster
  ☆12Updated last year
• wangsiping97 / FastGEMV
  High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.
  ☆81Updated 2 months ago
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆41Updated 3 weeks ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆36Updated 8 months ago