Alternatives and similar repositories for Fourier-Position-Embedding:

Users that are interested in Fourier-Position-Embedding are comparing it to the libraries listed below

• zhixuan-lin / forgetting-transformer
  [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"
  ☆95Updated 2 weeks ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆28Updated 10 months ago
• thu-ml / ReMoE
  [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆67Updated 4 months ago
• OpenSparseLLMs / Linearization
  ☆39Updated last month
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆26Updated 7 months ago
• RobertCsordas / moeut
  ☆78Updated 8 months ago
• Infini-AI-Lab / S2FT
  ☆17Updated 3 months ago
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆48Updated last month
• g-luo / task_vectors_are_cross_modal
  Official PyTorch Implementation for Task Vectors are Cross-Modal
  ☆22Updated 4 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆52Updated last month
• OpenSparseLLMs / MoM
  ☆74Updated this week
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆46Updated 2 months ago
• BestAnHongjun / SentenceVAE
  Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
  ☆31Updated 8 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆60Updated last year
• dllm-reasoning / d1
  Official Implementation for the paper "d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning"
  ☆77Updated last week
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆27Updated last year
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆54Updated 8 months ago
• juzhengz / LoRI
  Official PyTorch implementation of LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation.
  ☆13Updated 2 weeks ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆26Updated last year
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆37Updated 6 months ago
• VITA-Group / Ms-PoE
  "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…
  ☆29Updated 11 months ago
• assafbk / DeciMamba
  DeciMamba: Exploring the Length Extrapolation Potential of Mamba (ICLR 2025)
  ☆26Updated 2 weeks ago
• sustcsonglin / linear-attention-and-beyond-slides
  ☆69Updated 2 months ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated 11 months ago
• lucidrains / multimodal-dit-pytorch
  Implementation of a multimodal diffusion transformer in Pytorch
  ☆101Updated 10 months ago
• HazyResearch / prefix-linear-attention
  ☆53Updated 9 months ago
• AnonymousAlethiometer / SGD_SaI
  Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"
  ☆51Updated 2 months ago
• hkust-nlp / PreSelect
  ☆46Updated last month
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆82Updated 6 months ago
• nreHieW / minVAR
  Minimal Implementation of Visual Autoregressive Modelling (VAR)
  ☆30Updated last month