insoochung / transformer_bcq

Related projects: ⓘ

• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• naver-ai / model-stock
  Model Stock: All we need is just a few fine-tuned models
  ☆75Updated 5 months ago
• pytorch-labs / superblock
  A block oriented training approach for inference time optimization.
  ☆26Updated last month
• Doraemonzzz / Awesome-Triton-Resources
  Awesome Triton Resources
  ☆16Updated 3 weeks ago
• sustcsonglin / mamba-triton
  ☆42Updated 7 months ago
• kaistAI / GAP
  [ACL 2023] Gradient Ascent Post-training Enhances Language Model Generalization
  ☆27Updated last week
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆66Updated 7 months ago
• lucidrains / pause-transformer
  Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…
  ☆50Updated 10 months ago
• cloneofsimo / ezmup
  Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam
  ☆66Updated last month
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆101Updated last year
• DeepAuto-AI / hip-attention
  Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.
  ☆14Updated this week
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆34Updated 2 months ago
• epfml / DenseFormer
  ☆73Updated 5 months ago
• cloneofsimo / fim-llama-deepspeed
  ☆31Updated 8 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆100Updated last month
• shikaiqiu / compute-better-spent
  ☆42Updated 3 months ago
• kyegomez / MultiQueryAttention
  This is a simple torch implementation of the high performance Multi-Query Attention
  ☆15Updated last year
• gauss5930 / iDUS
  An unofficial implementation of SOLAR-10.7B model and the newly proposed interlocked-DUS(iDUS) implementation and experiment details.
  ☆10Updated 6 months ago
• knotgrass / attention
  several types of attention modules written in PyTorch
  ☆37Updated 4 months ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆57Updated 5 months ago
• daemyung / practice-triton
  삼각형의 실전! Triton
  ☆14Updated 7 months ago
• conceptofmind / flash-gpt
  ☆34Updated this week
• srush / mamba-primer
  ☆35Updated 5 months ago
• kaistAI / LangBridge
  [ACL 2024] LangBridge: Multilingual Reasoning Without Multilingual Supervision
  ☆63Updated last week
• srush / LLM-Talk
  ☆45Updated 7 months ago
• catie-aq / flashT5
  A fast implementation of T5/UL2 in PyTorch using Flash Attention
  ☆60Updated this week
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆55Updated 4 months ago
• microsoft / mutransformers
  some common Huggingface transformers in maximal update parametrization (µP)
  ☆76Updated 2 years ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 9 months ago