luyug / GradCacheLinks
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
☆399Updated last year
Alternatives and similar repositories for GradCache
Users that are interested in GradCache are comparing it to the libraries listed below
Sorting:
- Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"☆456Updated last year
- Research code for pixel-based encoders of language (PIXEL)☆337Updated 3 weeks ago
- Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models☆142Updated 2 years ago
- Code for the ALiBi method for transformer language models (ICLR 2022)☆538Updated last year
- Repository containing code for "How to Train BERT with an Academic Budget" paper☆314Updated last year
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆139Updated last year
- An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi☆270Updated 2 years ago
- MEND: Fast Model Editing at Scale☆249Updated last year
- Understanding the Difficulty of Training Transformers☆329Updated 3 years ago
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆132Updated last year
- Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings"☆294Updated 2 years ago
- ☆321Updated 4 years ago
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆276Updated 2 years ago
- ☆347Updated 3 years ago
- Train Dense Passage Retriever (DPR) with a single GPU☆133Updated 4 years ago
- Scalable training for dense retrieval models.☆299Updated last month
- Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)☆462Updated 2 years ago
- DSIR large-scale data selection framework for language model training☆258Updated last year
- Package to compute Mauve, a similarity score between neural text and human text. Install with `pip install mauve-text`.☆294Updated last year
- Implementation of "The Power of Scale for Parameter-Efficient Prompt Tuning"☆167Updated 3 years ago
- [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408☆196Updated 2 years ago
- Fusion-in-Decoder☆580Updated last year
- contrastive decoding☆203Updated 2 years ago
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆112Updated 2 years ago
- ICML'2022: Black-Box Tuning for Language-Model-as-a-Service & EMNLP'2022: BBTv2: Towards a Gradient-Free Future with Large Language Model…☆270Updated 2 years ago
- The original Backpack Language Model implementation, a fork of FlashAttention☆69Updated 2 years ago
- Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03…☆544Updated last year
- [NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation☆471Updated last year
- Search Engines with Autoregressive Language models☆291Updated 2 years ago
- Code repository for the paper - "Matryoshka Representation Learning"☆534Updated last year