eth-easl / fmengine

Related projects: ⓘ

• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• softmax1 / Flash-Attention-Softmax-N
  CUDA and Triton implementations of Flash Attention with SoftmaxN.
  ☆66Updated 3 months ago
• renll / SeqBoat
  [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling
  ☆34Updated 9 months ago
• lucidrains / memory-editable-transformer
  My explorations into editing the knowledge and memories of an attention network
  ☆34Updated last year
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆53Updated 5 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆28Updated last week
• cloneofsimo / min-fsdp
  ☆68Updated 2 months ago
• lucidrains / ReST-EM-pytorch
  ☆42Updated this week
• hadasah / btm
  ☆69Updated 4 months ago
• srush / LLM-Talk
  ☆45Updated 7 months ago
• ahennequ / pytorch-custom-mma
  ☆29Updated last year
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆62Updated 2 weeks ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆48Updated last week
• huggingface / peft-pytorch-conference
  Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…
  ☆14Updated 11 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 9 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆55Updated 4 months ago
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆34Updated 10 months ago
• RobertCsordas / moeut
  ☆50Updated last month
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆43Updated 3 weeks ago
• Zyphra / Zyda_processing
  ☆22Updated 3 months ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆33Updated last year
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆42Updated 3 weeks ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆66Updated 7 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆36Updated 8 months ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆28Updated 4 months ago
• lucidrains / pause-transformer
  Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…
  ☆50Updated 10 months ago
• catie-aq / flashT5
  A fast implementation of T5/UL2 in PyTorch using Flash Attention
  ☆60Updated this week
• euclaise / supertrainer2000
  ☆48Updated 6 months ago
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆58Updated last year