Alternatives and similar repositories for GAU-alpha

Users that are interested in GAU-alpha are comparing it to the libraries listed below

• JunnYu / FLASHQuad_pytorch
  FLASHQuad_pytorch
  ☆68Updated 3 years ago
• ZhuiyiTechnology / roformer-v2
  RoFormer升级版
  ☆154Updated 3 years ago
• nengwp / Lion-vs-Adam
  Lion and Adam optimization comparison
  ☆64Updated 2 years ago
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆126Updated last year
• 1140310118 / tdlm
  实现了Transformer中的几种位置编码方案
  ☆44Updated 4 years ago
• TsinghuaAI / CUGE
  ☆54Updated 3 years ago
• keezen / ntk_alibi
  NTK scaled version of ALiBi position encoding in Transformer.
  ☆69Updated 2 years ago
• bojone / LST-CLUE
  Ladder Side-Tuning在CLUE上的简单尝试
  ☆22Updated 3 years ago
• TobiasLee / Awesome-Efficient-PLM
  Must-read papers on improving efficiency for pre-trained language models.
  ☆105Updated 3 years ago
• YJiangcm / Lion
  [EMNLP 2023] Lion: Adversarial Distillation of Proprietary Large Language Models
  ☆212Updated last year
• JunnYu / RoFormer_pytorch
  RoFormer V1 & V2 pytorch
  ☆516Updated 3 years ago
• Hzfinfdu / PLMTuningCompetition
  擂台赛3-大规模预训练调优比赛的示例代码与baseline实现
  ☆37Updated 3 years ago
• bojone / r-drop
  R-Drop方法在中文任务上的简单实验
  ☆91Updated 3 years ago
• princeton-nlp / CoFiPruning
  [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
  ☆197Updated 2 years ago
• bojone / shuffle
  Python下shuffle几百G文件
  ☆33Updated 4 years ago
• seukgcode / FastRE
  The code and dataset for "FastRE: Towards Fast Relation Extraction with Convolutional Encoder and Improved Cascade Binary Tagging Framewo…
  ☆24Updated 3 years ago
• bojone / tiger
  A Tight-fisted Optimizer
  ☆50Updated 2 years ago
• CLUEbenchmark / SuperCLUE-Math6
  SuperCLUE-Math6：新一代中文原生多轮多步数学推理数据集的探索之旅
  ☆60Updated last year
• RUCKBReasoning / GLM-Dialog
  ☆59Updated 2 years ago
• TsinghuaAI / CPM-2-Finetune
  Finetune CPM-2
  ☆82Updated 2 years ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• wxl1999 / PLMPapers
  A paper list of pre-trained language models (PLMs).
  ☆81Updated 3 years ago
• TsinghuaAI / CPM-1-Pretrain
  Pretrain CPM-1
  ☆53Updated 4 years ago
• WENGSYX / CirBERTa
  Apply the Circular to the Pretraining Model
  ☆38Updated 3 years ago
• bojone / univae
  基于Transformer的单模型、多尺度的VAE模型
  ☆57Updated 4 years ago
• HillZhang1999 / CTC-Report
  CTC2021-中文文本纠错大赛的SOTA方案及在线演示
  ☆73Updated 2 years ago
• CLUEbenchmark / ZeroCLUE
  零样本学习测评基准，中文版
  ☆57Updated 4 years ago
• TsinghuaAI / CPM
  Introduction to CPM
  ☆166Updated 4 years ago
• beichao1314 / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆67Updated 2 years ago
• thu-coai / TaiLr
  ICLR2023 - Tailoring Language Generation Models under Total Variation Distance
  ☆21Updated 2 years ago