ZixuanJiang / pre-rmsnorm-transformer

Related projects ⓘ

Alternatives and complementary repositories for pre-rmsnorm-transformer

• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆72Updated 10 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆35Updated 4 months ago
• MatX-inc / seqax
  seqax = sequence modeling + JAX
  ☆133Updated 4 months ago
• spcl / sten
  Sparsity support for PyTorch
  ☆31Updated this week
• athms / mad-lab
  A MAD laboratory to improve AI architecture designs 🧪
  ☆95Updated 6 months ago
• google-research / jaxpruner
  ☆207Updated 6 months ago
• JeremieMelo / NeurOLight
  A Neural Operator-based Integrated Photonic Device Simulation Framework, NeurOLight NeurIPS 2022
  ☆34Updated last year
• siboehm / ShallowSpeed
  Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
  ☆107Updated last year
• insuhan / hyper-attn
  ☆74Updated 11 months ago
• lucidrains / taylor-series-linear-attention
  Explorations into the recently proposed Taylor Series Linear Attention
  ☆90Updated 3 months ago
• zinccat / Awesome-Triton-Kernels
  Collection of kernels written in Triton language
  ☆68Updated 3 weeks ago
• HazyResearch / butterfly
  Butterfly matrix multiplication in PyTorch
  ☆164Updated last year
• Vivswan / AnalogVNN
  A fully modular framework for modeling and optimizing analog neural networks
  ☆17Updated last month
• proger / accelerated-scan
  Accelerated First Order Parallel Associative Scan
  ☆164Updated 3 months ago
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆185Updated last month
• google / aqt
  ☆268Updated this week
• stas00 / ml-ways
  ML/DL Math and Method notes
  ☆57Updated 11 months ago
• DS3Lab / CocktailSGD
  ☆24Updated last year
• sholtodouglas / scalingExperiments
  ☆57Updated 2 years ago
• srush / anynp
  Proof-of-concept of global switching between numpy/jax/pytorch in a library.
  ☆18Updated 5 months ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• HanGuo97 / flute
  Fast Matrix Multiplications for Lookup Table-Quantized LLMs
  ☆187Updated this week
• hahnyuan / PB-LLM
  PB-LLM: Partially Binarized Large Language Models
  ☆148Updated last year
• HazyResearch / flash-fft-conv
  FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
  ☆281Updated last month
• microsoft / mutransformers
  some common Huggingface transformers in maximal update parametrization (µP)
  ☆76Updated 2 years ago
• alexzhang13 / Triton-Puzzles-Solutions
  Personal solutions to the Triton Puzzles
  ☆16Updated 4 months ago
• google-deepmind / dks
  Multi-framework implementation of Deep Kernel Shaping and Tailored Activation Transformations, which are methods that modify neural netwo…
  ☆64Updated this week
• gpu-mode / triton-index
  Cataloging released Triton kernels.
  ☆138Updated 2 months ago
• srush / do-we-need-attention
  ☆161Updated last year
• epfml / dynamic-sparse-flash-attention
  ☆132Updated last year