luyug / GradCacheLinks
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
☆424Updated last year
Alternatives and similar repositories for GradCache
Users that are interested in GradCache are comparing it to the libraries listed below
Sorting:
- Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"☆457Updated 2 years ago
- Research code for pixel-based encoders of language (PIXEL)☆345Updated 6 months ago
- Code for the ALiBi method for transformer language models (ICLR 2022)☆549Updated 2 years ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆151Updated 2 years ago
- Scalable training for dense retrieval models.☆298Updated 7 months ago
- Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models☆143Updated 3 years ago
- MEND: Fast Model Editing at Scale☆258Updated 2 years ago
- An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi☆273Updated 2 years ago
- Train Dense Passage Retriever (DPR) with a single GPU☆136Updated 4 years ago
- Code repository for the paper - "Matryoshka Representation Learning"☆594Updated last year
- Understanding the Difficulty of Training Transformers☆332Updated 3 years ago
- ☆351Updated 4 years ago
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆136Updated 2 years ago
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆112Updated 2 years ago
- Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03…☆553Updated 2 years ago
- Search Engines with Autoregressive Language models☆295Updated 2 years ago
- Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings"☆296Updated 3 years ago
- Flexible components pairing 🤗 Transformers with Pytorch Lightning☆612Updated 3 years ago
- ☆130Updated 3 years ago
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆285Updated 3 years ago
- A huggingface transformers implementation of "Transformer Memory as a Differentiable Search Index"☆176Updated 3 years ago
- Princeton NLP's pre-training library based on fairseq with DeepSpeed kernel integration 🚃☆116Updated 3 years ago
- ☆329Updated 4 years ago
- Fusion-in-Decoder☆590Updated 2 years ago
- DSIR large-scale data selection framework for language model training☆269Updated last year
- Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning☆768Updated 2 years ago
- contrastive decoding☆206Updated 3 years ago
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆235Updated 2 years ago
- Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)☆330Updated 2 years ago
- [NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation☆474Updated last year