lucidrains / electra-pytorch

Related projects ⓘ

Alternatives and complementary repositories for electra-pytorch

• richarddwang / electra_pytorch
  Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)
  ☆325Updated 10 months ago
• lucidrains / mlm-pytorch
  An implementation of masked language modeling for Pytorch, made as concise and simple as possible
  ☆177Updated last year
• lucidrains / charformer-pytorch
  Implementation of the GBST block from the Charformer paper, in Pytorch
  ☆117Updated 3 years ago
• LiyuanLucasLiu / Transformer-Clinic
  Understanding the Difficulty of Training Transformers
  ☆328Updated 2 years ago
• bloodwass / mixout
  Implementation of Mixout with PyTorch
  ☆74Updated last year
• yang-zhang / lightning-language-modeling
  Language Modeling Example with Transformers and PyTorch Lighting
  ☆65Updated 4 years ago
• IntelLabs / academic-budget-bert
  Repository containing code for "How to Train BERT with an Academic Budget" paper
  ☆309Updated last year
• laiguokun / Funnel-Transformer
  ☆212Updated 4 years ago
• uds-lsv / bert-stable-fine-tuning
  On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
  ☆132Updated last year
• luyug / GradCache
  Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
  ☆361Updated 7 months ago
• voidism / DiffCSE
  Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings"
  ☆291Updated 2 years ago
• Lightning-Universe / lightning-transformers
  Flexible components pairing 🤗 Transformers with Pytorch Lightning
  ☆611Updated last year
• archinetai / smart-pytorch
  PyTorch – SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models.
  ☆59Updated 2 years ago
• lucidrains / routing-transformer
  Fully featured implementation of Routing Transformer
  ☆284Updated 3 years ago
• SeanNaren / minGPT
  A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!
  ☆111Updated last year
• huanghonggit / Mask-Language-Model
  pytorch； mask language model ； bert
  ☆71Updated 4 years ago
• gsarti / lambda-bert
  A 🤗-style implementation of BERT using lambda layers instead of self-attention
  ☆70Updated 4 years ago
• facebookresearch / mega
  Sequence modeling with Mega.
  ☆298Updated last year
• ricardorei / lightning-text-classification
  Minimalist implementation of a BERT Sentence Classifier with PyTorch Lightning, Transformers and PyTorch-NLP.
  ☆215Updated last year
• OATML / RHO-Loss
  ☆187Updated 2 years ago
• microsoft / xtreme-distil-transformers
  XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale
  ☆153Updated 11 months ago
• urvashik / knnlm
  ☆315Updated 3 years ago
• lucidrains / compressive-transformer-pytorch
  Pytorch implementation of Compressive Transformers, from Deepmind
  ☆157Updated 3 years ago
• ofirpress / attention_with_linear_biases
  Code for the ALiBi method for transformer language models (ICLR 2022)
  ☆507Updated last year
• r-three / t-few
  Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"
  ☆431Updated last year
• timoschick / dino
  This repository contains the code for "Generating Datasets with Pretrained Language Models".
  ☆187Updated 3 years ago
• huggingface / olm-training
  Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.
  ☆92Updated last year
• lucidrains / feedback-transformer-pytorch
  Implementation of Feedback Transformer in Pytorch
  ☆104Updated 3 years ago
• lucidrains / h-transformer-1d
  Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning
  ☆155Updated 9 months ago
• tnq177 / transformers_without_tears
  Transformers without Tears: Improving the Normalization of Self-Attention
  ☆130Updated 5 months ago