Alternatives and similar repositories for softpick-attention:

Users that are interested in softpick-attention are comparing it to the libraries listed below

• zaydzuhri / flame
  Fork of Flame repo for training of some new stuff in development
  ☆11Updated last week
• cloneofsimo / zeroshampoo
  ☆33Updated 7 months ago
• riverstone496 / awesome-second-order-optimization
  ☆26Updated last year
• cloneofsimo / efae
  ☆22Updated 10 months ago
• fal-ai / diffusion-speedrun
  Focused on fast experimentation and simplicity
  ☆71Updated 4 months ago
• fal-ai-community / NativeSparseAttention
  research impl of Native Sparse Attention (2502.11089)
  ☆53Updated 2 months ago
• cloneofsimo / min-fsdp
  ☆78Updated 10 months ago
• ml-gde / jflux
  JAX Implementation of Black Forest Labs' Flux.1 family of models
  ☆31Updated 6 months ago
• edwardmilsom / function-space-learning-rates-paper
  Code for the paper "Function-Space Learning Rates"
  ☆19Updated 2 weeks ago
• dvruette / barrel-rec-pytorch
  ☆53Updated last year
• VatsaDev / NanoPoor
  NanoGPT-speedrunning for the poor T4 enjoyers
  ☆63Updated 2 weeks ago
• lucidrains / firefly-torch
  Exploration into the Firefly algorithm in Pytorch
  ☆38Updated 2 months ago
• lucidrains / quartic-transformer
  Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)
  ☆50Updated last month
• ethansmith2000 / fsdp_optimizers
  supporting pytorch FSDP for optimizers
  ☆80Updated 4 months ago
• EleutherAI / training-jacobian
  ☆21Updated 4 months ago
• evanatyourservice / llm-jax
  Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.
  ☆17Updated last month
• fal-ai-community / minDDPD
  ☆34Updated 4 months ago
• catid / lllm
  Latent Large Language Models
  ☆18Updated 8 months ago
• Adamdad / neumeta
  NeuMeta transforms neural networks by allowing a single model to adapt on the fly to different sizes, generating the right weights when n…
  ☆42Updated 5 months ago
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆37Updated last year
• CLAIRE-Labo / EvoTune
  Efficiently discovering algorithms via LLMs with evolutionary search and reinforcement learning.
  ☆68Updated 2 weeks ago
• euclaise / supertrainer2000
  ☆49Updated last year
• joey00072 / Multi-Head-Latent-Attention-MLA-
  working implimention of deepseek MLA
  ☆41Updated 3 months ago
• facebookresearch / capi
  Code and weights for the paper "Cluster and Predict Latents Patches for Improved Masked Image Modeling"
  ☆101Updated 3 weeks ago
• evanatyourservice / kron_torch
  An implementation of PSGD Kron second-order optimizer for PyTorch
  ☆89Updated last month
• SHI-Labs / CompactNet
  ☆31Updated last year
• jfpuget / ARC-AGI-Challenge-2024
  ☆52Updated 5 months ago
• lucidrains / GAF-microbatch-pytorch
  Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch
  ☆24Updated 3 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆27Updated 7 months ago
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆49Updated last month