Alternatives and similar repositories for Derf

Users that are interested in Derf are comparing it to the libraries listed below

• galilai-group / llm-jepa
  ☆169Updated 4 months ago
• JinjieNi / dlms-are-super-data-learners
  The official github repo for "Diffusion Language Models are Super Data Learners".
  ☆219Updated 2 months ago
• zaydzuhri / softpick-attention
  Landing repository for the paper "Softpick: No Attention Sink, No Massive Activations with Rectified Softmax"
  ☆86Updated 4 months ago
• tokenbender / mHC-manifold-constrained-hyper-connections
  implementations and experimentation on mHC by deepseek - https://arxiv.org/abs/2512.24880
  ☆278Updated 3 weeks ago
• facebookresearch / Mixture-of-Transformers
  Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models. TMLR 2025.
  ☆140Updated 4 months ago
• ShadeAlsha / ICon
  ICLR 2025 - official implementation for "I-Con: A Unifying Framework for Representation Learning"
  ☆126Updated 7 months ago
• HanGuo97 / log-linear-attention
  ☆269Updated 7 months ago
• ChenWu98 / algorithmic-creativity
  [ICML 2025] Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
  ☆84Updated 8 months ago
• RWKV / RWKV-LM
  RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best…
  ☆59Updated 10 months ago
• martin-marek / batch-size
  📄Small Batch Size Training for Language Models
  ☆80Updated 3 months ago
• jacobfa / fft
  ☆129Updated 5 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆137Updated last month
• sail-sg / Precision-RL
  Defeating the Training-Inference Mismatch via FP16
  ☆180Updated 2 months ago
• riverstone496 / awesome-second-order-optimization
  ☆27Updated 4 months ago
• alexiglad / EBT
  PyTorch Code for Energy-Based Transformers paper -- generalizable reasoning and scalable learning
  ☆584Updated 2 months ago
• apple / ml-l3m
  Large multi-modal models (L3M) pre-training.
  ☆229Updated 4 months ago
• apple / ml-sigmoid-attention
  ☆306Updated 9 months ago
• model-architectures / GRAPE
  Official implementation of GRAPE: Group Representational Position Encoding (https://arxiv.org/abs/2512.07805)
  ☆74Updated 3 weeks ago
• s-sahoo / Eso-LMs
  Esoteric Language Models
  ☆110Updated 2 months ago
• ESHyperscale / HyperscaleES
  Jax Codebase for Evolutionary Strategies at the Hyperscale
  ☆216Updated last month
• yifanzhang-pro / deep-delta-learning
  Official Project Page for Deep Delta Learning (https://huggingface.co/papers/2601.00417)
  ☆320Updated this week
• Zyphra / Zamba2
  PyTorch implementation of models from the Zamba2 series.
  ☆186Updated last year
• wmn-231314 / diffusion-data-constraint
  Official PyTorch implementation and models for paper "Diffusion Beats Autoregressive in Data-Constrained Settings". We find diffusion mod…
  ☆119Updated 3 weeks ago
• thinking-machines-lab / manifolds
  Supporting code for the blog post on modular manifolds.
  ☆115Updated 4 months ago
• lucidrains / vit-arc-slot
  Explorations into improving ViTArc with Slot Attention
  ☆43Updated last year
• google-deepmind / tips
  TIPS (ICLR'25): Text-Image Pretraining with Spatial Awareness
  ☆115Updated 9 months ago
• NVIDIA / ngpt
  Normalized Transformer (nGPT)
  ☆198Updated last year
• apple / ml-pararnn
  ☆162Updated 3 months ago
• bluorion-com / ZClip
  Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".
  ☆144Updated 2 months ago
• lucidrains / PEER-pytorch
  Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
  ☆134Updated 3 months ago