Alternatives and similar repositories for Hedgehog:

Users that are interested in Hedgehog are comparing it to the libraries listed below

• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆25Updated 9 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆53Updated 7 months ago
• katiekang1998 / reasoning_generalization
  ☆30Updated 2 months ago
• formll / resolving-scaling-law-discrepancies
  ☆18Updated 8 months ago
• Qichuzyy / POA
  Official implementation of ECCV24 paper: POA
  ☆24Updated 7 months ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆26Updated 11 months ago
• caojiaolong / Awesome-Mamba
  Collect papers about Mamba (a selective state space model).
  ☆14Updated 7 months ago
• pixeli99 / MixLN
  Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxiang Li, Lu Yi…
  ☆19Updated 3 months ago
• bdusell / stack-attention
  Code for the paper "Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns"
  ☆17Updated last year
• kiddyboots216 / lottery-ticket-adaptation
  Lottery Ticket Adaptation
  ☆38Updated 4 months ago
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆23Updated 8 months ago
• HazyResearch / embroid
  Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
  ☆11Updated last year
• qiuzh20 / RMoE
  Official implementation of RMoE (Layerwise Recurrent Router for Mixture-of-Experts)
  ☆18Updated 7 months ago
• nverma1 / merging-text-transformers
  Code for "Merging Text Transformers from Different Initializations"
  ☆19Updated last month
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆24Updated 4 months ago
• uscnlp-lime / safer-instruct
  This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"
  ☆17Updated last year
• uclaml / PDE
  Official repo of Progressive Data Expansion: data, code and evaluation
  ☆28Updated last year
• BBuf / flash-rwkv
  ☆30Updated 9 months ago
• SprocketLab / sparse_matrix_fine_tuning
  Official repository for ICML 2024 paper "MoRe Fine-Tuning with 10x Fewer Parameters"
  ☆17Updated last week
• facebookresearch / adaptive_scheduling
  Experimental scripts for researching data adaptive learning rate scheduling.
  ☆23Updated last year
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆30Updated last week
• JerryYin777 / Cross-Layer-Attention
  Self Reproduction Code of Paper "Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (MIT CSAIL)
  ☆12Updated 10 months ago
• IBM / ColPret
  Efficient Scaling laws and collaborative pretraining.
  ☆15Updated last month
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 9 months ago
• kyegomez / MultiQueryAttention
  This is a simple torch implementation of the high performance Multi-Query Attention
  ☆16Updated last year
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆60Updated 10 months ago
• szc12153 / sparse_meta_tuning
  Official implementation for Sparse MetA-Tuning (SMAT)
  ☆16Updated 8 months ago