apd10 / RzLinear

Related projects: ⓘ

• apd10 / universal_memory_allocation
  ☆14Updated 2 years ago
• facebookresearch / MODel_opt
  Memory Optimizations for Deep Learning (ICML 2023)
  ☆58Updated 6 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆34Updated 2 months ago
• microsoft / DeepSpeed-Kernels
  ☆50Updated 3 months ago
• aredden / torch-cublas-hgemm
  PyTorch half precision gemm lib w/ fused optional bias + optional relu/gelu
  ☆25Updated 3 weeks ago
• sustcsonglin / gated_linear_attention_layer
  ☆30Updated 8 months ago
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆23Updated 9 months ago
• stanford-futuredata / stk
  ☆83Updated 3 weeks ago
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆22Updated 3 months ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• Raincleared-Song / DejaVu_predictor
  The codes for training sparsity predictor on LLaMA.
  ☆14Updated 4 months ago
• VITA-Group / Structure-LTH
  [ICML 2022] "Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets" by Tianlong Chen, Xuxi Chen, Xiaolong Ma, Yanzhi Wa…
  ☆30Updated last year
• huyz2023 / 2by4-pretrain
  Efficient 2:4 sparse training algorithms and implementations
  ☆18Updated 3 months ago
• CerebrasResearch / Sparse-IFT
  Official repository of Sparse ISO-FLOP Transformations for Maximizing Training Efficiency
  ☆23Updated last month
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆55Updated 4 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆47Updated 2 weeks ago
• TalSchuster / CATs
  Confident Adaptive Transformers
  ☆12Updated 3 years ago
• mobiusml / gemlite
  Simple and fast low-bit matmul kernels in CUDA
  ☆48Updated this week
• rlin27 / DeBut
  Codes of the paper Deformable Butterfly: A Highly Structured and Sparse Linear Transform.
  ☆12Updated 2 years ago
• sanagno / adaptively_sparse_attention
  ☆16Updated last year
• tridao / flash-attention-wheels
  ☆38Updated 9 months ago
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆56Updated last week
• emalach / LinearLM
  Code for the paper: https://arxiv.org/pdf/2309.06979.pdf
  ☆14Updated last month
• tridao / cutlass_quant
  ☆16Updated this week
• Dao-AILab / fast-hadamard-transform
  Fast Hadamard transform in CUDA, with a PyTorch interface
  ☆87Updated 3 months ago
• codekansas / rwkv
  RWKV model implementation
  ☆38Updated last year
• Ryu1845 / hyena-jax
  Implementation of Hyena Hierarchy in JAX
  ☆10Updated last year
• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• tobiaskatsch / GatedLinearRNN
  ☆23Updated 6 months ago
• kssteven418 / SqueezeLLM-gradients
  ☆14Updated 7 months ago