BlinkDL / minGPT-tunedLinks
A *tuned* minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
☆115Updated 3 years ago
Alternatives and similar repositories for minGPT-tuned
Users that are interested in minGPT-tuned are comparing it to the libraries listed below
Sorting:
- 基于Transformer的单模型、多尺度的VAE模型☆57Updated 4 years ago
- Source code for paper: Knowledge Inheritance for Pre-trained Language Models☆38Updated 3 years ago
- Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch☆46Updated 4 years ago
- Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"☆71Updated 2 years ago
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆57Updated last year
- Code for the paper "A Theoretical Analysis of the Repetition Problem in Text Generation" in AAAI 2021.☆54Updated 2 years ago
- A pre-trained model with multi-exit transformer architecture.☆54Updated 2 years ago
- The code of paper "Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation" published at NeurIPS 202…☆46Updated 2 years ago
- A small framework mimics PyTorch using CuPy or NumPy☆41Updated 3 years ago
- ☆50Updated 2 years ago
- A PyTorch implementation of the paper - "Synthesizer: Rethinking Self-Attention in Transformer Models"☆73Updated 2 years ago
- Transformers at any scale☆41Updated last year
- ☆97Updated last year
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆82Updated 3 years ago
- Code for the paper "Query-Key Normalization for Transformers"☆43Updated 4 years ago
- Python下shuffle几百G文件☆33Updated 3 years ago
- An Experiment on Dynamic NTK Scaling RoPE☆64Updated last year
- A variant of Transformer-XL where the memory is updated not with a queue, but with attention☆49Updated 4 years ago
- Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch☆70Updated 5 years ago
- A pytorch &keras implementation and demo of Fastformer.☆189Updated 2 years ago
- Implementation of RealFormer using pytorch☆100Updated 4 years ago
- Implementation of pQRNN in PyTorch☆46Updated 3 years ago
- FairSeq repo with Apollo optimizer☆114Updated last year
- PALM: Pre-training an Autoencoding & Autoregressive Language Model for Context-conditioned Generation☆34Updated 2 years ago
- codes and pre-trained models of paper "Segatron: Segment-aware Transformer for Language Modeling and Understanding"☆18Updated 2 years ago
- The multilingual variant of GLM, a general language model trained with autoregressive blank infilling objective☆62Updated 2 years ago
- RWKV-v2-RNN trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.☆67Updated 2 years ago
- Finetune CPM-1☆24Updated 4 years ago
- 基于Gated Attention Unit的Transformer模型（尝鲜版）☆98Updated 2 years ago
- Code for "Mixed Cross Entropy Loss for Neural Machine Translation"☆20Updated 3 years ago