Phylliida / MambaLens

Related projects ⓘ

Alternatives and complementary repositories for MambaLens

• RobertCsordas / moeut
  ☆62Updated 3 months ago
• berlino / seq_icl
  ☆50Updated 6 months ago
• JacobPfau / fillerTokens
  ☆49Updated 6 months ago
• sustcsonglin / mamba-triton
  ☆45Updated 9 months ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆56Updated last month
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆56Updated 6 months ago
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆61Updated 7 months ago
• ScalingIntelligence / large_language_monkeys
  ☆55Updated last month
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆34Updated last week
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 7 months ago
• hughbzhang / o1_inference_scaling_laws
  Replicating O1 inference-time scaling laws
  ☆49Updated last month
• GindaChen / FlexFlashAttention3
  FlexAttention w/ FlashAttention3 Support
  ☆27Updated last month
• epfml / schedules-and-scaling
  ☆53Updated 3 weeks ago
• Ryu1845 / hyena-jax
  Implementation of Hyena Hierarchy in JAX
  ☆10Updated last year
• PiotrNawrot / nano-sparse-attention
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆37Updated this week
• wesg52 / universal-neurons
  Universal Neurons in GPT2 Language Models
  ☆27Updated 5 months ago
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆61Updated last month
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆39Updated 4 months ago
• nanowell / Q-Sparse-LLM
  My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
  ☆30Updated 3 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆40Updated 2 months ago
• maximzubkov / fft-scan
  Efficient PScan implementation in PyTorch
  ☆15Updated 10 months ago
• JoshEngels / MultiDimensionalFeatures
  Code for reproducing our paper "Not All Language Model Features Are Linear"
  ☆61Updated last week
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 11 months ago
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆43Updated 4 months ago
• abhishekpanigrahi1996 / transformer_in_transformer
  ☆44Updated last year
• jysohn1108 / Looped-Transformer
  Official implementation of the transformer (TF) architecture suggested in a paper entitled "Looped Transformers as Programmable Computers…
  ☆22Updated last year
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆84Updated this week
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆72Updated 10 months ago
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆50Updated 7 months ago
• sjelassi / transformers_ssm_copy
  ☆24Updated 8 months ago