Qualcomm-AI-research / llm-surgeon

Related projects: ⓘ

• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆77Updated last year
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆104Updated 6 months ago
• JonasGeiping / linear_cross_entropy_loss
  A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.
  ☆48Updated last month
• epfml / llm-baselines
  ☆68Updated last month
• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• shikaiqiu / compute-better-spent
  ☆42Updated 3 months ago
• epfml / schedules-and-scaling
  ☆47Updated 3 months ago
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆91Updated 5 months ago
• goombalab / phi-mamba
  Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…
  ☆61Updated last week
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆62Updated 11 months ago
• srush / mamba-primer
  ☆35Updated 5 months ago
• TianjinYellow / EdgeDeviceLLMCompetition-Starting-Kit
  ☆38Updated 2 weeks ago
• MadryLab / datamodels-data
  Data for "Datamodels: Predicting Predictions with Training Data"
  ☆87Updated last year
• softmax1 / Flash-Attention-Softmax-N
  CUDA and Triton implementations of Flash Attention with SoftmaxN.
  ☆66Updated 3 months ago
• alexzhang13 / flashattention2-custom-mask
  Triton implementation of FlashAttention2 that adds Custom Masks.
  ☆62Updated last month
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆51Updated 3 months ago
• hwang595 / Cuttlefish
  The implementation for MLSys 2023 paper: "Cuttlefish: Low-rank Model Training without All The Tuning"
  ☆42Updated last year
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆63Updated 3 months ago
• gregorbachmann / scaling_mlps
  ☆48Updated 3 months ago
• prateeky2806 / ties-merging
  ☆136Updated 7 months ago
• MadryLab / DsDm
  ☆43Updated 7 months ago
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆99Updated 6 months ago
• insuhan / hyper-attn
  ☆65Updated 9 months ago
• varunnair18 / FISH
  Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).
  ☆54Updated 2 years ago
• HanGuo97 / lq-lora
  ☆117Updated 7 months ago
• UCDvision / NOLA
  Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"
  ☆46Updated 3 weeks ago
• berlino / seq_icl
  ☆48Updated 4 months ago
• epfml / dynamic-sparse-flash-attention
  ☆129Updated last year
• nateraw / huggingface-vit-finetune
  Finetune Google's pre-trained ViT models from HuggingFace's model hub.
  ☆18Updated 3 years ago
• cloneofsimo / min-fsdp
  ☆68Updated 2 months ago