Alternatives and similar repositories for GAU-alpha:

Users that are interested in GAU-alpha are comparing it to the libraries listed below

• JunnYu / FLASHQuad_pytorch
  FLASHQuad_pytorch
  ☆67Updated 3 years ago
• ZhuiyiTechnology / roformer-v2
  RoFormer升级版
  ☆152Updated 2 years ago
• 1140310118 / tdlm
  实现了Transformer中的几种位置编码方案
  ☆40Updated 3 years ago
• keezen / ntk_alibi
  NTK scaled version of ALiBi position encoding in Transformer.
  ☆67Updated last year
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆119Updated last year
• xsysigma / TencentLLMEval
  TencentLLMEval is a comprehensive and extensive benchmark for artificial evaluation of large models that includes task trees, standards, …
  ☆38Updated 2 weeks ago
• nengwp / Lion-vs-Adam
  Lion and Adam optimization comparison
  ☆60Updated 2 years ago
• bojone / LST-CLUE
  Ladder Side-Tuning在CLUE上的简单尝试
  ☆19Updated 2 years ago
• TsinghuaAI / CUGE
  ☆53Updated 2 years ago
• bojone / tiger
  A Tight-fisted Optimizer
  ☆47Updated 2 years ago
• wxl1999 / PLMPapers
  A paper list of pre-trained language models (PLMs).
  ☆80Updated 3 years ago
• bojone / r-drop
  R-Drop方法在中文任务上的简单实验
  ☆91Updated 3 years ago
• JunnYu / RoFormer_pytorch
  RoFormer V1 & V2 pytorch
  ☆491Updated 2 years ago
• CLUEbenchmark / SuperCLUE-Math6
  SuperCLUE-Math6：新一代中文原生多轮多步数学推理数据集的探索之旅
  ☆53Updated last year
• bojone / P-tuning
  P-tuning方法在中文上的简单实验
  ☆139Updated 3 years ago
• TsinghuaAI / CPM-1-Pretrain
  Pretrain CPM-1
  ☆51Updated 3 years ago
• thu-coai / TaiLr
  ICLR2023 - Tailoring Language Generation Models under Total Variation Distance
  ☆21Updated 2 years ago
• xv44586 / Chinese-instruction-datasets
  中文 Instruction tuning datasets
  ☆129Updated 11 months ago
• princeton-nlp / CoFiPruning
  [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
  ☆196Updated last year
• LydiaXiaohongLi / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆19Updated last year
• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆151Updated 2 years ago
• bojone / CLUE-bert4keras
  真 · “Deep Learning for Humans”
  ☆141Updated 3 years ago
• TsinghuaAI / CPM-2-Finetune
  Finetune CPM-2
  ☆82Updated 2 years ago
• bojone / rerope
  Rectified Rotary Position Embeddings
  ☆361Updated 10 months ago
• taishan1994 / pytorch-distributed-NLP
  pytorch分布式训练
  ☆64Updated last year
• Macielyoung / sentence_representation_matching
  句子匹配模型，包括无监督的SimCSE、ESimCSE、PromptBERT，和有监督的SBERT、CoSENT。
  ☆98Updated 2 years ago
• muyaostudio / qwen2_seq_cls
  使用 Qwen2ForSequenceClassification 简单实现文本分类任务。
  ☆59Updated 9 months ago
• JunnYu / WoBERT_pytorch
  WoBERT_pytorch
  ☆40Updated 3 years ago
• TobiasLee / Awesome-Efficient-PLM
  Must-read papers on improving efficiency for pre-trained language models.
  ☆103Updated 2 years ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆72Updated last year