MadryLab / modelcomponents

Related projects: ⓘ

• uclaml / MoE
  Towards Understanding the Mixture-of-Experts Layer in Deep Learning
  ☆19Updated 9 months ago
• AmeenAli / HiddenMambaAttn
  Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
  ☆186Updated 3 months ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆104Updated 6 months ago
• multimodal-interpretability / maia
  Official implementation of MAIA, A Multimodal Automated Interpretability Agent
  ☆56Updated last month
• goombalab / phi-mamba
  Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…
  ☆61Updated this week
• Adamdad / kat
  Kolmogorov-Arnold Transformer: A PyTorch Implementation with CUDA kernel
  ☆221Updated this week
• koayon / awesome-adaptive-computation
  A curated reading list of research in Adaptive Computation, Dynamic Compute & Mixture of Experts (MoE). Inference time compute as seen in…
  ☆123Updated last month
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆153Updated last week
• goombalab / hydra
  Official implementation of "Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers"
  ☆94Updated last month
• prateeky2806 / ties-merging
  ☆136Updated 7 months ago
• ycjing / Awesome-Model-Merging
  A curated list of Model Merging methods.
  ☆71Updated this week
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆62Updated 11 months ago
• KihoPark / linear_rep_geometry
  ☆68Updated 7 months ago
• shikaiqiu / compute-better-spent
  ☆42Updated 3 months ago
• franciscoliu / Awesome-GenAI-Unlearning
  ☆57Updated last week
• oripress / EntropyEnigma
  Official code for the ICML 2024 paper "The Entropy Enigma: Success and Failure of Entropy Minimization"
  ☆44Updated 3 months ago
• HazyResearch / zoology
  Understand and test language model architectures on synthetic tasks.
  ☆156Updated 4 months ago
• JoshEngels / MultiDimensionalFeatures
  Code for reproducing our paper "Not All Language Model Features Are Linear"
  ☆57Updated last week
• kyegomez / Jamba
  PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"
  ☆120Updated last week
• hyperevolnet / Terminator
  The official repository for HyperZ⋅Z⋅W Operator Connects Slow-Fast Networks for Full Context Interaction.
  ☆29Updated this week
• lucidrains / mirasol-pytorch
  Implementation of 🌻 Mirasol, SOTA Multimodal Autoregressive model out of Google Deepmind, in Pytorch
  ☆87Updated 8 months ago
• pengzhangzhi / Awesome-Mamba
  Awesome list of papers that extend Mamba to various applications.
  ☆124Updated 2 weeks ago
• Leiay / looped_transformer
  ☆18Updated 9 months ago
• KellerJordan / cifar10-airbench
  94% on CIFAR-10 in 3.09 seconds 💨 96% in 27 seconds
  ☆127Updated last month
• apple / ml-sigmoid-attention
  ☆190Updated last week
• bfshi / TOAST
  Official code for "TOAST: Transfer Learning via Attention Steering"
  ☆186Updated last year
• thjashin / multires-conv
  Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)
  ☆119Updated 11 months ago
• epfml / llm-baselines
  ☆68Updated 3 weeks ago
• stanislavfort / dissect-git-re-basin
  Replicating and dissecting the git-re-basin project in one-click-replication Colabs
  ☆36Updated 2 years ago
• jxiw / MambaInLlama
  Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models
  ☆130Updated this week