luyug / GradCache
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
☆359Updated 7 months ago
Related projects ⓘ
Alternatives and complementary repositories for GradCache
- Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"☆430Updated last year
- Repository containing code for "How to Train BERT with an Academic Budget" paper☆309Updated last year
- Code for the ALiBi method for transformer language models (ICLR 2022)☆506Updated last year
- Research code for pixel-based encoders of language (PIXEL)☆330Updated 8 months ago
- Scalable training for dense retrieval models.☆270Updated last year
- MEND: Fast Model Editing at Scale☆234Updated last year
- An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi☆253Updated last year
- Search Engines with Autoregressive Language models☆277Updated last year
- [NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation☆466Updated 8 months ago
- Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings"☆290Updated 2 years ago
- Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models☆133Updated 2 years ago
- ☆315Updated 3 years ago
- Code repository for the paper - "Matryoshka Representation Learning"☆423Updated 8 months ago
- ☆339Updated 3 years ago
- contrastive decoding☆178Updated last year
- Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch☆224Updated 2 months ago
- Sequence modeling with Mega.☆297Updated last year
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆126Updated last year
- DSIR large-scale data selection framework for language model training☆227Updated 7 months ago
- Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)☆457Updated 2 years ago
- Train Dense Passage Retriever (DPR) with a single GPU☆128Updated 3 years ago
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆111Updated last year
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆269Updated 2 years ago
- Tevatron - A flexible toolkit for neural retrieval research and development.☆517Updated 2 weeks ago
- Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch☆291Updated 4 months ago
- ☆293Updated last year
- ☆126Updated 2 years ago
- Implementation of paper "Towards a Unified View of Parameter-Efficient Transfer Learning" (ICLR 2022)☆517Updated 2 years ago
- [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408☆191Updated last year
- Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)☆324Updated 9 months ago