luyug / GradCache
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
☆382Updated last year
Alternatives and similar repositories for GradCache:
Users that are interested in GradCache are comparing it to the libraries listed below
- Code for the ALiBi method for transformer language models (ICLR 2022)☆520Updated last year
- Code for the NAACL 2022 long paper "DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings"☆293Updated 2 years ago
- Repository containing code for "How to Train BERT with an Academic Budget" paper☆312Updated last year
- Flexible components pairing 🤗 Transformers with Pytorch Lightning☆609Updated 2 years ago
- Research code for pixel-based encoders of language (PIXEL)☆334Updated last year
- Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models☆140Updated 2 years ago
- Code repository for the paper - "Matryoshka Representation Learning"☆474Updated last year
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆225Updated last year
- Search Engines with Autoregressive Language models☆284Updated last year
- Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch☆226Updated 6 months ago
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆270Updated 2 years ago
- ☆318Updated 3 years ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆131Updated last year
- Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"☆447Updated last year
- Scalable training for dense retrieval models.☆284Updated last month
- Train Dense Passage Retriever (DPR) with a single GPU☆130Updated 3 years ago
- Tevatron - A flexible toolkit for neural retrieval research and development.☆580Updated this week
- Approximate nearest neighbor search with product quantization on GPU in pytorch and cuda☆221Updated last year
- Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)☆327Updated last year
- Implementation of paper "Towards a Unified View of Parameter-Efficient Transfer Learning" (ICLR 2022)☆523Updated 3 years ago
- Understanding the Difficulty of Training Transformers☆328Updated 2 years ago
- [NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation☆471Updated last year
- Fusion-in-Decoder☆564Updated last year
- MEND: Fast Model Editing at Scale☆242Updated last year
- An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi☆262Updated last year
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆130Updated last year
- Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch☆860Updated last year
- Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets☆317Updated last year
- A huggingface transformers implementation of "Transformer Memory as a Differentiable Search Index"☆169Updated 2 years ago
- [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408☆196Updated last year