luyug / GradCacheLinks
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
☆393Updated last year
Alternatives and similar repositories for GradCache
Users that are interested in GradCache are comparing it to the libraries listed below
Sorting:
- Research code for pixel-based encoders of language (PIXEL)☆335Updated last year
- Code for the ALiBi method for transformer language models (ICLR 2022)☆530Updated last year
- Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"☆451Updated last year
- Fusion-in-Decoder☆570Updated last year
- Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings"☆294Updated 2 years ago
- Flexible components pairing 🤗 Transformers with Pytorch Lightning☆609Updated 2 years ago
- Search Engines with Autoregressive Language models☆286Updated 2 years ago
- Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch☆230Updated 9 months ago
- Repository containing code for "How to Train BERT with an Academic Budget" paper☆313Updated last year
- An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi☆263Updated 2 years ago
- Train Dense Passage Retriever (DPR) with a single GPU☆131Updated 3 years ago
- Tevatron - Unified Document Retrieval Toolkit across Scale, Language, and Modality. Demo in SIGIR 2023, SIGIR 2025.☆613Updated 2 weeks ago
- Scalable training for dense retrieval models.☆292Updated 3 months ago
- Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models☆142Updated 2 years ago
- Understanding the Difficulty of Training Transformers☆329Updated 3 years ago
- ☆319Updated 3 years ago
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆227Updated last year
- Sequence modeling with Mega.☆295Updated 2 years ago
- [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408☆195Updated 2 years ago
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆273Updated 2 years ago
- [NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation☆471Updated last year
- contrastive decoding☆201Updated 2 years ago
- DSIR large-scale data selection framework for language model training☆249Updated last year
- Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)☆329Updated last year
- A huggingface transformers implementation of "Transformer Memory as a Differentiable Search Index"☆172Updated 2 years ago
- Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning☆737Updated 2 years ago
- ☆354Updated last year
- EMNLP 2021 - Pre-training architectures for dense retrieval☆252Updated 3 years ago
- Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch☆866Updated last year
- ☆291Updated 2 years ago