Alternatives and similar repositories for flash-attention

Users that are interested in flash-attention are comparing it to the libraries listed below

• openai / consistency_models_cifar10
  Consistency models trained on CIFAR-10, in JAX.
  ☆154Updated 2 years ago
• LAION-AI / Open-GIA
  O-GIA is an umbrella for research, infrastructure and projects ecosystem that should provide open source, reproducible datasets, models, …
  ☆89Updated 2 years ago
• petals-infra / health.petals.dev
  🏥 Health monitor for a Petals swarm
  ☆39Updated last year
• geohot / dumbrl
  Can RL solve simple problems?
  ☆54Updated last year
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆119Updated last year
• The-Swarm-Corporation / Mamba-R1
  Mamba R1 represents a novel architecture that combines the efficiency of Mamba's state space models with the scalability of Mixture of Ex…
  ☆25Updated last month
• facebookresearch / DIG-In
  This library supports evaluating disparities in generated image quality, diversity, and consistency between geographic regions.
  ☆20Updated last year
• microsoft / gps-china-democenter
  Show Azure Openai demos for partners
  ☆23Updated 2 years ago
• SmerkyG / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆12Updated last month
• cchan / nanoGPT-fp8
  ☆13Updated 2 years ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆44Updated this week
• lucidrains / llama-qrlhf
  Implementation of the Llama architecture with RLHF + Q-learning
  ☆168Updated 9 months ago
• moomou / listening-with-llm
  ☆17Updated last year
• lucidrains / PaLM-jax
  Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)
  ☆188Updated 3 years ago
• sfcompute / tinynarrations
  A synthetic story narration dataset to study small audio LMs.
  ☆31Updated last year
• chai-research / lmgym
  Code base for internal reward models and PPO training
  ☆24Updated 2 years ago
• sekstini / basedxl
  ☆18Updated last year
• evanatyourservice / llm-jax
  Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.
  ☆18Updated 4 months ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆86Updated 7 months ago
• mlfoundations / open-diffusion
  Simple large-scale training of stable diffusion with multi-node support.
  ☆133Updated 2 years ago
• tensoic / Cerule
  Cerule - A Tiny Mighty Vision Model
  ☆67Updated 2 weeks ago
• ayaka14732 / llama-2-jax
  JAX implementation of the Llama 2 model
  ☆216Updated last year
• lucidrains / taylor-series-linear-attention
  Explorations into the recently proposed Taylor Series Linear Attention
  ☆100Updated last year
• idiap / sigma-gpt
  σ-GPT: A New Approach to Autoregressive Models
  ☆69Updated last year
• lucidrains / maskbit-pytorch
  Implementation of the proposed MaskBit from Bytedance AI
  ☆82Updated last year
• lucidrains / TPDNE
  Thispersondoesnotexist went down, so this time, while building it back up, I am going to open source all of it.
  ☆91Updated 2 years ago
• bhosmer / mm
  ☆138Updated last year
• AI-Hypercomputer / maxdiffusion
  ☆277Updated this week
• JoeLi12345 / nGPT
  an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)
  ☆107Updated 8 months ago
• NousResearch / StripedHyenaTrainer
  ☆62Updated last year