gordicaleksa / OpenGemini

Related projects ⓘ

Alternatives and complementary repositories for OpenGemini

• kyegomez / OpenStrawberry
  An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO
  ☆20Updated last week
• lucidrains / quartic-transformer
  Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)
  ☆43Updated last month
• kyegomez / MobileVLM
  Implementation of the LDP module block in PyTorch and Zeta from the paper: "MobileVLM: A Fast, Strong and Open Vision Language Assistant …
  ☆14Updated 8 months ago
• lucidrains / kalman-filtering-attention
  Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"
  ☆57Updated last year
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆84Updated 2 months ago
• lucidrains / firefly-torch
  Exploration into the Firefly algorithm in Pytorch
  ☆35Updated last month
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆53Updated 5 months ago
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆39Updated 3 months ago
• cloneofsimo / zeroshampoo
  ☆31Updated 2 months ago
• lucidrains / scaling-vin-pytorch
  Exploration into the Scaling Value Iteration Networks paper, from Schmidhuber's group
  ☆36Updated last month
• SonicCodes / subcloning
  implementation of https://arxiv.org/pdf/2312.09299
  ☆19Updated 4 months ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• ethansmith2000 / TransformerExperiments
  ☆18Updated last month
• lucidrains / taylor-series-linear-attention
  Explorations into the recently proposed Taylor Series Linear Attention
  ☆89Updated 2 months ago
• catid / spectral_ssm
  Implementation of Spectral State Space Models
  ☆17Updated 8 months ago
• RodkinIvan / associative-recurrent-memory-transformer
  [ICML 24 NGSM workshop] Associative Recurrent Memory Transformer implementation and scripts for training and evaluating
  ☆31Updated this week
• Zyphra / zcookbook
  Training hybrid models for dummies.
  ☆15Updated 2 weeks ago
• BlinkDL / SmallInitEmb
  LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence
  ☆45Updated 2 years ago
• lucaslingle / mu_transformer
  Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.
  ☆29Updated last week
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆108Updated 3 weeks ago
• EleutherAI / rnngineering
  Engineering the state of RNN language models (Mamba, RWKV, etc.)
  ☆32Updated 5 months ago
• lucidrains / coordinate-descent-attention
  Implementation of an Attention layer where each head can attend to more than just one token, using coordinate descent to pick topk
  ☆46Updated last year
• hyperevolnet / Terminator
  The official repository for HyperZ⋅Z⋅W Operator Connects Slow-Fast Networks for Full Context Interaction.
  ☆31Updated last month
• lucidrains / frame-averaging-pytorch
  Pytorch implementation of a simple way to enable (Stochastic) Frame Averaging for any network
  ☆46Updated 3 months ago
• kyegomez / MM1
  PyTorch Implementation of the paper "MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training"
  ☆23Updated last week
• nanowell / Q-Sparse-LLM
  My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
  ☆30Updated 2 months ago
• lucidrains / vit-arc-slot
  Explorations into improving ViTArc with Slot Attention
  ☆36Updated 3 weeks ago
• tridao / flash-attention-wheels
  ☆42Updated 11 months ago