state-spaces / s4

Related projects ⓘ

Alternatives and complementary repositories for s4

• idiap / fast-transformers
  Pytorch library for fast transformer implementations
  ☆1,642Updated last year
• lucidrains / lion-pytorch
  🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
  ☆2,028Updated 4 months ago
• srush / annotated-s4
  Implementation of https://srush.github.io/annotated-s4
  ☆468Updated last year
• lucidrains / vector-quantize-pytorch
  Vector (and Scalar) Quantization, in Pytorch
  ☆2,594Updated this week
• lucidrains / linear-attention-transformer
  Transformer based on a variant of attention that is linear complexity in respect to sequence length
  ☆695Updated 6 months ago
• microsoft / mup
  maximal update parametrization (µP)
  ☆1,398Updated 3 months ago
• google-research / long-range-arena
  Long Range Arena for Benchmarking Efficient Transformers
  ☆727Updated 10 months ago
• alxndrTL / mamba.py
  A simple and efficient Mamba implementation in pure PyTorch and MLX.
  ☆1,004Updated 2 months ago
• lucidrains / x-transformers
  A concise but complete full-attention transformer with a set of promising experimental features from various papers
  ☆4,768Updated this week
• lucidrains / rotary-embedding-torch
  Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch
  ☆565Updated last month
• ashleve / lightning-hydra-template
  PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡
  ☆4,236Updated 2 months ago
• lucidrains / performer-pytorch
  An implementation of Performer, a linear attention-based transformer, in Pytorch
  ☆1,093Updated 2 years ago
• Lightning-AI / torchmetrics
  Machine learning metrics for distributed, scalable PyTorch applications.
  ☆2,133Updated this week
• johnma2006 / mamba-minimal
  Simple, minimal implementation of the Mamba SSM in one file of PyTorch.
  ☆2,614Updated 8 months ago
• davidmrau / mixture-of-experts
  PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538
  ☆974Updated 6 months ago
• lucidrains / perceiver-pytorch
  Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch
  ☆1,092Updated last year
• google / learned_optimization
  ☆749Updated last month
• xl0 / lovely-tensors
  Tensors, for human consumption
  ☆1,111Updated last week
• facebookresearch / schedule_free
  Schedule-Free Optimization in PyTorch
  ☆1,880Updated this week
• facebookresearch / multimodal
  TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.
  ☆1,460Updated this week
• atong01 / conditional-flow-matching
  TorchCFM: a Conditional Flow Matching library
  ☆1,198Updated last month
• state-spaces / mamba
  Mamba SSM architecture
  ☆13,130Updated this week
• microsoft / torchscale
  Foundation Architecture for (M)LLMs
  ☆3,025Updated 6 months ago
• Jamie-Stirling / RetNet
  An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"
  ☆1,163Updated last year
• webdataset / webdataset
  A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.
  ☆2,303Updated last month
• yang-song / score_sde_pytorch
  PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)
  ☆1,736Updated 3 months ago
• radarFudan / Awesome-state-space-models
  Collection of papers on state-space models
  ☆549Updated this week
• sustcsonglin / flash-linear-attention
  Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
  ☆1,320Updated this week
• lucidrains / reformer-pytorch
  Reformer, the efficient Transformer, in Pytorch
  ☆2,116Updated last year
• clementchadebec / benchmark_VAE
  Unifying Variational Autoencoder (VAE) implementations in Pytorch (NeurIPS 2022)
  ☆1,816Updated 3 months ago