Alternatives and similar repositories for Mixture-of-Experts-Sentence-Similarity

Users that are interested in Mixture-of-Experts-Sentence-Similarity are comparing it to the libraries listed below

• Alab-NII / Awesome-SciLM
  Pre-trained Language Model for Scientific Text
  ☆45Updated last year
• zwhe99 / LLM-MT-Eval
  {DeepL, Google, WMT-Best, davinci-003, turbo, gpt-4} × {En-De, En-Cs, En-Ru, En-Zh, De-Fr, En-Ja, Uk-En, Uk-Cs, En-Hr, En-Ha, En-Is}
  ☆14Updated 2 years ago
• FranxYao / Retrieval-Head-with-Flash-Attention
  Efficient retrieval head analysis with triton flash attention that supports topK probability
  ☆13Updated last year
• yangjackie / Topics-on-diffusion-generative-models
  ☆28Updated 3 months ago
• Yuanhy1997 / SeqDiffuSeq
  Text Diffusion Model with Encoder-Decoder Transformers for Sequence-to-Sequence Generation [NAACL 2024]
  ☆98Updated 2 years ago
• ProtLLM / ProtLLM
  [ACL 2024] ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training
  ☆50Updated last year
• alon-albalak / FLAD
  Few-shot Learning with Auxiliary Data
  ☆31Updated 2 years ago
• thoppe / The-Pile-PubMed
  Download, parse, and filter data PubMed, data-ready for The-Pile
  ☆23Updated 4 years ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆56Updated 11 months ago
• sanyalsunny111 / Early_Weight_Avg
  [COLM 2024] Early Weight Averaging meets High Learning Rates for LLM Pre-training
  ☆18Updated last year
• theblackcat102 / unify-learning-paradigms
  data collator for UL2 and U-PaLM
  ☆29Updated 2 years ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆89Updated last year
• ducdauge / sft-llm
  Scaling Sparse Fine-Tuning to Large Language Models
  ☆18Updated last year
• catie-aq / flashT5
  A fast implementation of T5/UL2 in PyTorch using Flash Attention
  ☆112Updated 2 months ago
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆48Updated last year
• AntNLP / nope_head_scale
  ☆29Updated last year
• yifeiwang77 / Self-Correction
  ☆20Updated last year
• piotrpiekos / MoSA
  User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…
  ☆28Updated 8 months ago
• princeton-pli / MeCo
  Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆48Updated 6 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆36Updated last year
• gersteinlab / BioCoder
  BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Models https://arxiv.org/abs/2308.16458
  ☆53Updated 5 months ago
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆40Updated last year
• lucidrains / equiformer-diffusion
  Implementation of Denoising Diffusion for protein design, but using the new Equiformer (successor to SE3 Transformers) with some addition…
  ☆57Updated 3 years ago
• TianduoWang / DiffAug
  [EMNLP 2022] Differentiable Data Augmentation for Contrastive Sentence Representation Learning. https://arxiv.org/abs/2210.16536
  ☆40Updated 3 years ago
• lsj2408 / URPE
  [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)
  ☆34Updated 2 years ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆56Updated 2 years ago
• THUDM / SciGLM
  SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning (NeurIPS D&B Track 2024)
  ☆86Updated last year
• dptech-corp / Uni-SMART
  ☆52Updated last year
• Namco0816 / PT-BERT
  ACL 2022(findings): A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings
  ☆18Updated 3 years ago
• allenai / EmbeddingRecycling
  Embedding Recycling for Language models
  ☆38Updated 2 years ago