JonasGeiping / linear_cross_entropy_loss

Related projects: ⓘ

• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆91Updated 5 months ago
• berlino / seq_icl
  ☆48Updated 4 months ago
• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• MadryLab / DsDm
  ☆43Updated 7 months ago
• gregorbachmann / Next-Token-Failures
  ☆61Updated 6 months ago
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆53Updated 5 months ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆77Updated last year
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆99Updated 6 months ago
• epfml / schedules-and-scaling
  ☆47Updated 3 months ago
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆25Updated 5 months ago
• sustcsonglin / mamba-triton
  ☆42Updated 7 months ago
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆51Updated 3 months ago
• logix-project / logix
  AI Logging for Interpretability and Explainability🔬
  ☆74Updated 3 months ago
• allenai / bff
  ☆38Updated 5 months ago
• epfml / dynamic-sparse-flash-attention
  ☆129Updated last year
• McGill-NLP / length-generalization
  Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023
  ☆123Updated 4 months ago
• epfml / llm-baselines
  ☆68Updated 3 weeks ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore, paper coming soon
  ☆18Updated this week
• ahans30 / goldfish-loss
  Official implementation of Goldfish Loss: Mitigating Memorization in Generative LLMs
  ☆68Updated 2 months ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆57Updated 5 months ago
• HazyResearch / zoology
  Understand and test language model architectures on synthetic tasks.
  ☆156Updated 4 months ago
• google-deepmind / randomized_positional_encodings
  Randomized Positional Encodings Boost Length Generalization of Transformers
  ☆78Updated 6 months ago
• insuhan / hyper-attn
  ☆65Updated 9 months ago
• wellecks / lm-evaluation-harness
  A framework for few-shot evaluation of autoregressive language models.
  ☆23Updated 8 months ago
• RobertCsordas / moeut
  ☆50Updated last month
• alexzhang13 / flashattention2-custom-mask
  Triton implementation of FlashAttention2 that adds Custom Masks.
  ☆62Updated last month
• abhishekpanigrahi1996 / transformer_in_transformer
  ☆44Updated 11 months ago
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆73Updated last month
• berlino / gated_linear_attention
  ☆94Updated 6 months ago
• hadasah / btm
  ☆69Updated 4 months ago