Alternatives and similar repositories for An-Empirical-Model-of-Large-Batch-Training:

Users that are interested in An-Empirical-Model-of-Large-Batch-Training are comparing it to the libraries listed below

• maximzubkov / fft-scan
  Efficient PScan implementation in PyTorch
  ☆15Updated last year
• berlino / seq_icl
  ☆51Updated 7 months ago
• sustcsonglin / mamba-triton
  ☆46Updated 11 months ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆25Updated 9 months ago
• HazyResearch / prefix-linear-attention
  ☆47Updated 6 months ago
• sjelassi / transformers_ssm_copy
  ☆29Updated 10 months ago
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆66Updated 9 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆41Updated last week
• srush / mamba-primer
  ☆37Updated 9 months ago
• chijames / KERPLE
  ☆18Updated 2 years ago
• renll / SeqBoat
  [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling
  ☆35Updated last year
• young-geng / mintext
  Minimal but scalable implementation of large language models in JAX
  ☆28Updated 2 months ago
• NVlabs / GatedDeltaNet
  Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆76Updated 2 weeks ago
• nanowell / Q-Sparse-LLM
  My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
  ☆31Updated 5 months ago
• Leiay / looped_transformer
  ☆22Updated last year
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆62Updated 8 months ago
• IdoAmos / not-from-scratch
  ☆26Updated 2 months ago
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆96Updated 9 months ago
• wesg52 / universal-neurons
  Universal Neurons in GPT2 Language Models
  ☆27Updated 7 months ago
• Zanette-Labs / SpeculativeRejection
  [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection
  ☆38Updated 2 months ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆58Updated 3 months ago
• kyo-takano / chinchilla
  A toolkit for scaling law research ⚖
  ☆43Updated last month
• epfml / schedules-and-scaling
  Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"
  ☆67Updated 2 months ago
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆56Updated 3 months ago
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆52Updated 9 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆52Updated 5 months ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆41Updated 5 months ago
• radarFudan / mamba-minimal-jax
  ☆31Updated last month
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆79Updated last year
• Nix07 / finetuning
  This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity…
  ☆21Updated 9 months ago