Alternatives and similar repositories for Mixture-of-Experts-Sentence-Similarity

Users that are interested in Mixture-of-Experts-Sentence-Similarity are comparing it to the libraries listed below

• FranxYao / Retrieval-Head-with-Flash-Attention
  Efficient retrieval head analysis with triton flash attention that supports topK probability
  ☆13Updated last year
• yangjackie / Topics-on-diffusion-generative-models
  ☆27Updated 2 months ago
• Alab-NII / Awesome-SciLM
  Pre-trained Language Model for Scientific Text
  ☆46Updated last year
• Yuanhy1997 / SeqDiffuSeq
  Text Diffusion Model with Encoder-Decoder Transformers for Sequence-to-Sequence Generation [NAACL 2024]
  ☆98Updated 2 years ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆36Updated last year
• ducdauge / sft-llm
  Scaling Sparse Fine-Tuning to Large Language Models
  ☆18Updated last year
• theblackcat102 / unify-learning-paradigms
  data collator for UL2 and U-PaLM
  ☆29Updated 2 years ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆89Updated last year
• dptech-corp / Uni-SMART
  ☆51Updated last year
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆56Updated 10 months ago
• sanyalsunny111 / Early_Weight_Avg
  [COLM 2024] Early Weight Averaging meets High Learning Rates for LLM Pre-training
  ☆18Updated last year
• alon-albalak / FLAD
  Few-shot Learning with Auxiliary Data
  ☆31Updated 2 years ago
• vvvm23 / sundae
  Unofficial PyTorch implementation of "Step-unrolled Denoising Autoencoders for Text Generation"
  ☆24Updated 3 years ago
• microsoft / ResiDual
  ResiDual: Transformer with Dual Residual Connections, https://arxiv.org/abs/2304.14802
  ☆96Updated 2 years ago
• googleinterns / localizing-paragraph-memorization
  ☆15Updated last year
• instance-wise-ordered-transformer / IOT
  ☆20Updated 4 years ago
• RUCAIBox / BAMBOO
  ☆35Updated last year
• wzq016 / PINE
  Offcial Repo of Paper "Eliminating Position Bias of Language Models: A Mechanistic Approach""
  ☆19Updated 6 months ago
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆48Updated last year
• thoppe / The-Pile-PubMed
  Download, parse, and filter data PubMed, data-ready for The-Pile
  ☆23Updated 4 years ago
• princeton-pli / MeCo
  Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆48Updated 5 months ago
• SciMT / SciMT-benchmark
  ☆11Updated last year
• expz / annotated-hyena
  An annotated implementation of the Hyena Hierarchy paper
  ☆34Updated 2 years ago
• chijames / KERPLE
  ☆20Updated 3 years ago
• AntNLP / nope_head_scale
  ☆29Updated last year
• allenai / EmbeddingRecycling
  Embedding Recycling for Language models
  ☆38Updated 2 years ago
• piotrpiekos / MoSA
  User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…
  ☆28Updated 7 months ago
• sunyt32 / torchscale
  Transformers at any scale
  ☆42Updated last year
• luohongyin / UniLC
  Interpretable unified language safety checking with large language models
  ☆31Updated 2 years ago
• gl-ybnbxb / BoNBoN
  ☆18Updated last year