Alternatives and similar repositories for MambaPEFT:

Users that are interested in MambaPEFT are comparing it to the libraries listed below

• facebookresearch / adaptive_scheduling
  Experimental scripts for researching data adaptive learning rate scheduling.
  ☆23Updated last year
• kyegomez / MultiQueryAttention
  This is a simple torch implementation of the high performance Multi-Query Attention
  ☆16Updated last year
• GindaChen / FlexFlashAttention3
  FlexAttention w/ FlashAttention3 Support
  ☆26Updated 6 months ago
• graphcore-research / jax-scalify
  JAX Scalify: end-to-end scaled arithmetics
  ☆16Updated 5 months ago
• srush / tangent
  Source-to-Source Debuggable Derivatives in Pure Python
  ☆15Updated last year
• kyegomez / MM1
  PyTorch Implementation of the paper "MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training"
  ☆23Updated last week
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆47Updated last year
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆37Updated last year
• Ryu1845 / hyena-jax
  Implementation of Hyena Hierarchy in JAX
  ☆10Updated last year
• catid / spectral_ssm
  Implementation of Spectral State Space Models
  ☆16Updated last year
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆43Updated 7 months ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆44Updated last year
• AndPotap / einsum-search
  ☆32Updated 6 months ago
• lucidrains / GAF-microbatch-pytorch
  Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch
  ☆24Updated 3 months ago
• f-dangel / vivit
  [TMLR 2022] Curvature access through the generalized Gauss-Newton's low-rank structure: Eigenvalues, eigenvectors, directional derivative…
  ☆17Updated last year
• kuleshov-group / remdm
  Remasking Discrete Diffusion Models with Inference-Time Scaling
  ☆18Updated last month
• FrancescoSaverioZuppichini / pytorch-2.0-benchmark
  Benchmarking PyTorch 2.0 different models
  ☆21Updated 2 years ago
• ahennequ / pytorch-custom-mma
  ☆29Updated 2 years ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆30Updated 10 months ago
• Shalev-Lifshitz / MultiAgentVerification
  Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers
  ☆17Updated last month
• Percent-BFD / neurips_submission
  ☆15Updated last year
• kyegomez / Hedgehog
  Implementation of the model "Hedgehog" from the paper: "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry"
  ☆13Updated last year
• iwiwi / epochraft
  Checkpointable dataset utilities for foundation model training
  ☆32Updated last year
• evanatyourservice / llm-jax
  Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.
  ☆17Updated last month
• prateeky2806 / ComPEFT
  ☆25Updated last year
• SHI-Labs / CompactNet
  ☆31Updated 11 months ago
• EleutherAI / training-jacobian
  ☆21Updated 4 months ago
• PiotrNawrot / sparse-frontier
  Official implementation of "The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs"
  ☆21Updated this week
• lucidrains / coordinate-descent-attention
  Implementation of an Attention layer where each head can attend to more than just one token, using coordinate descent to pick topk
  ☆46Updated last year
• scalable-model-editing / unified-model-editing
  We introduce EMMET and unify model editing with popular algorithms ROME and MEMIT.
  ☆17Updated 4 months ago