HazyResearch / m2

Related projects ⓘ

Alternatives and complementary repositories for m2

• lucidrains / MEGABYTE-pytorch
  Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
  ☆624Updated last month
• haoliuhl / ringattention
  Transformers with Arbitrarily Large Context
  ☆637Updated 2 months ago
• srush / annotated-mamba
  Annotated version of the Mamba paper
  ☆455Updated 8 months ago
• HazyResearch / based
  Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
  ☆212Updated 2 months ago
• mlfoundations / open_lm
  A repository for research on medium sized language models.
  ☆479Updated this week
• lucidrains / ring-attention-pytorch
  Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
  ☆474Updated 2 weeks ago
• llm-random / llm-random
  ☆175Updated this week
• pratyushasharma / laser
  The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
  ☆368Updated 4 months ago
• dingo-actual / infini-transformer
  PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention…
  ☆279Updated 6 months ago
• HazyResearch / zoology
  Understand and test language model architectures on synthetic tasks.
  ☆161Updated 6 months ago
• lucidrains / st-moe-pytorch
  Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
  ☆291Updated 4 months ago
• lucidrains / recurrent-memory-transformer-pytorch
  Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch
  ☆393Updated 8 months ago
• microsoft / Samba
  Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"
  ☆801Updated 2 months ago
• jzhang38 / EasyContext
  Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.
  ☆642Updated last month
• RAIVNLab / MRL
  Code repository for the paper - "Matryoshka Representation Learning"
  ☆423Updated 8 months ago
• princeton-nlp / LLM-Shearing
  [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
  ☆553Updated 8 months ago
• datamllab / LongLM
  [ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
  ☆610Updated 5 months ago
• syncdoth / RetNet
  Huggingface compatible implementation of RetNet (Retentive Networks, https://arxiv.org/pdf/2307.08621.pdf) including parallel, recurrent,…
  ☆226Updated 7 months ago
• imoneoi / multipack_sampler
  Multipack distributed sampler for fast padding-free training of LLMs
  ☆175Updated 3 months ago
• huggingface / cosmopedia
  ☆445Updated last week
• tomaarsen / attention_sinks
  Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
  ☆667Updated 6 months ago
• microsoft / TransformerCompression
  For releasing code related to compression methods for transformers, accompanying our publications
  ☆369Updated 3 weeks ago
• huggingface / nanotron
  Minimalistic large language model 3D-parallelism training
  ☆1,227Updated this week
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆190Updated 9 months ago
• sustcsonglin / flash-linear-attention
  Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
  ☆1,320Updated this week
• SkunkworksAI / hydra-moe
  ☆411Updated last year
• yuhuixu1993 / qa-lora
  Official PyTorch implementation of QA-LoRA
  ☆116Updated 7 months ago
• XuezheMax / megalodon
  Reference implementation of Megalodon 7B model
  ☆504Updated 6 months ago
• jondurbin / bagel
  A bagel, with everything.
  ☆312Updated 6 months ago
• pytorch-labs / attention-gym
  Helpful tools and examples for working with flex-attention
  ☆460Updated 2 weeks ago