Alternatives and similar repositories for Data-Efficient-Scaling:

Users that are interested in Data-Efficient-Scaling are comparing it to the libraries listed below

• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 10 months ago
• Doraemonzzz / xmixers
  Xmixers: A collection of SOTA efficient token/channel mixers
  ☆11Updated 5 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆30Updated 10 months ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆27Updated last year
• sustcsonglin / mamba-triton
  ☆47Updated last year
• RUCAIBox / QuantizedEmpirical
  ☆14Updated last year
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆64Updated last year
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆51Updated 2 years ago
• HazyResearch / prefix-linear-attention
  ☆53Updated 9 months ago
• jungokasai / T2R
  ☆14Updated 2 years ago
• epfml / pam
  ☆15Updated last year
• IBM / selective-dense-state-space-model
  Open-sourcing code associated with the AAAI-25 paper "On the Expressiveness and Length Generalization of Selective State-Space Models on …
  ☆13Updated 2 months ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆16Updated 2 weeks ago
• abdelfattah-lab / TokenButler
  ☆22Updated last month
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• AntNLP / nope_head_scale
  ☆24Updated last year
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆44Updated 3 weeks ago
• sanagno / adaptively_sparse_attention
  ☆21Updated last year
• chijames / KERPLE
  ☆19Updated 2 years ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆54Updated 8 months ago
• VITA-Group / LoCoCo
  [ICML‘2024] "LoCoCo: Dropping In Convolutions for Long Context Compression", Ruisi Cai, Yuandong Tian, Zhangyang Wang, Beidi Chen
  ☆16Updated 7 months ago
• sail-sg / LongSpec
  LongSpec: Long-Context Speculative Decoding with Efficient Drafting and Verification
  ☆51Updated 2 months ago
• VITA-Group / Ms-PoE
  "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…
  ☆29Updated 11 months ago
• liutianlin0121 / decoding-time-realignment
  Implementation of "Decoding-time Realignment of Language Models", ICML 2024.
  ☆18Updated 10 months ago
• DRSY / KV_Compression
  [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens
  ☆24Updated last year
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆34Updated 7 months ago
• uservan / ThinkPO
  ☆20Updated 2 months ago
• wzq016 / PINE
  Offcial Repo of Paper "Eliminating Position Bias of Language Models: A Mechanistic Approach""
  ☆14Updated 8 months ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆44Updated 2 years ago
• twinkle0331 / LGTM
  [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…
  ☆38Updated last year