WangHuiNEU / Transformer_Knowlegde

Related projects: ⓘ

• simtony / mlrunner
  A light-weight script for maintaining a LOT of machine learning experiments.
  ☆88Updated last year
• ZhuiyiTechnology / GAU-alpha
  基于Gated Attention Unit的Transformer模型（尝鲜版）
  ☆95Updated last year
• infly-ai / INF-LLM
  The official repo of INF-34B models trained by INF Technology.
  ☆32Updated last month
• Outsider565 / LoRA-GA
  ☆125Updated last week
• serend1p1ty / core-pytorch-utils
  Yet another PyTorch Trainer and some core components for deep learning.
  ☆202Updated 4 months ago
• 1140310118 / tdlm
  实现了Transformer中的几种位置编码方案
  ☆36Updated 2 years ago
• bojone / tiger
  A Tight-fisted Optimizer
  ☆46Updated last year
• TobiasLee / Awesome-Efficient-PLM
  Must-read papers on improving efficiency for pre-trained language models.
  ☆100Updated last year
• firechecking / CleanTransformer
  an implementation of transformer, bert, gpt, and diffusion models for learning purposes
  ☆139Updated 10 months ago
• bobo0810 / LearnDeepSpeed
  DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）
  ☆94Updated last year
• WangHuiNEU / llm
  The Roadmap for LLMs
  ☆84Updated last year
• BIGBALLON / distribuuuu
  The pure and clear PyTorch Distributed Training Framework.
  ☆276Updated 7 months ago
• nengwp / Lion-vs-Adam
  Lion and Adam optimization comparison
  ☆56Updated last year
• transformer-vq / transformer_vq
  ☆170Updated 9 months ago
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆111Updated 6 months ago
• dada-qin / Data-Centric_LLM_Studies
  A list of papers about data quality in Large Language Models (LLMs)
  ☆18Updated 9 months ago
• JunnYu / FLASHQuad_pytorch
  FLASHQuad_pytorch
  ☆66Updated 2 years ago
• cofe-ai / Mu-scaling
  Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales
  ☆30Updated last year
• Chongjie-Si / Subspace-Tuning
  A generalized framework for subspace tuning methods in parameter efficient fine-tuning.
  ☆70Updated this week
• bojone / rerope
  Rectified Rotary Position Embeddings
  ☆329Updated 4 months ago
• cmnfriend / O-LoRA
  ☆139Updated 2 months ago
• ZhiyuanChen / CHANfiG
  Easier Configuration
  ☆30Updated last week
• princeton-nlp / CoFiPruning
  [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
  ☆188Updated last year
• IndexFziQ / Diffusion4NLP-Papers
  A paper list about diffusion models for natural language processing.
  ☆170Updated last year
• Ablustrund / LoRAMoE
  LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment
  ☆191Updated 4 months ago
• JunnYu / RoFormer_pytorch
  RoFormer V1 & V2 pytorch
  ☆462Updated 2 years ago
• keezen / ntk_alibi
  NTK scaled version of ALiBi position encoding in Transformer.
  ☆64Updated last year
• dongguanting / In-Context-Learning_PaperList
  Paper List for In-context Learning 🌷
  ☆165Updated 6 months ago
• OpenNLPLab / cosFormer
  [ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention
  ☆178Updated last year
• Spico197 / watchmen
  😎 A simple and easy-to-use toolkit for GPU scheduling.
  ☆40Updated 3 years ago