transformer-vq / transformer_vq

Related projects: ⓘ

• Chaos96 / fourierft
  ☆119Updated last week
• Outsider565 / LoRA-GA
  ☆125Updated this week
• OpenNLPLab / lightning-attention
  Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
  ☆182Updated 4 months ago
• nbasyl / DoRA
  Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
  ☆120Updated 4 months ago
• bojone / FSQ
  Keras implement of Finite Scalar Quantization
  ☆58Updated 10 months ago
• OpenNLPLab / TransnormerLLM
  Official implementation of TransNormerLLM: A Faster and Better LLM
  ☆223Updated 7 months ago
• berlino / gated_linear_attention
  ☆94Updated 6 months ago
• HKUNLP / efficient-attention
  [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling
  ☆78Updated last year
• fkodom / grouped-query-attention-pytorch
  (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …
  ☆116Updated 4 months ago
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆109Updated 11 months ago
• QingruZhang / AdaLoRA
  AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).
  ☆253Updated last year
• minyoungg / vqtorch
  ☆103Updated 6 months ago
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆95Updated 3 months ago
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆63Updated 3 months ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆86Updated 5 months ago
• wutaiqiang / MoSLoRA
  ☆54Updated 2 months ago
• kyegomez / AttentionIsOFFByOne
  Implementation of "Attention Is Off By One" by Evan Miller
  ☆177Updated last year
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆114Updated 2 months ago
• lucidrains / soft-moe-pytorch
  Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
  ☆233Updated 4 months ago
• GraphPKU / PiSSA
  PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
  ☆244Updated last month
• dvlab-research / Step-DPO
  Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"
  ☆225Updated 2 months ago
• OpenNLPLab / Transnormer
  [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer
  ☆53Updated last year
• ziqipang / LM4VisualEncoding
  [ICLR 2024 (Spotlight)] "Frozen Transformers in Language Models are Effective Visual Encoder Layers"
  ☆215Updated 8 months ago
• NUS-HPC-AI-Lab / InfoBatch
  Lossless Training Speed Up by Unbiased Dynamic Data Pruning
  ☆310Updated 2 weeks ago
• pengzhangzhi / Awesome-Mamba
  Awesome list of papers that extend Mamba to various applications.
  ☆124Updated 2 weeks ago
• AvivBick / awesome-ssm-ml
  Reading list for research topics in state-space models
  ☆209Updated last week
• bojone / rerope
  Rectified Rotary Position Embeddings
  ☆329Updated 4 months ago
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆111Updated 6 months ago
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆184Updated 7 months ago
• lucidrains / st-moe-pytorch
  Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
  ☆278Updated 3 months ago