Guitaricet / my_pefty_llama
Minimal implementation of multiple PEFT methods for LLaMA fine-tuning
☆13Updated last year
Alternatives and similar repositories for my_pefty_llama:
Users that are interested in my_pefty_llama are comparing it to the libraries listed below
- ☆46Updated 2 months ago
- ☆52Updated last year
- Embedding Recycling for Language models☆38Updated last year
- Aioli: A unified optimization framework for language model data mixing☆18Updated this week
- ☆15Updated 10 months ago
- Transformers at any scale☆41Updated last year
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆45Updated last year
- A extension of Transformers library to include T5ForSequenceClassification class.☆37Updated last year
- Exploring finetuning public checkpoints on filter 8K sequences on Pile☆115Updated last year
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆93Updated last year
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆171Updated last week
- SILO Language Models code repository☆81Updated 10 months ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆153Updated last month
- The Official Repository for "Bring Your Own Data! Self-Supervised Evaluation for Large Language Models"☆108Updated last year
- Truly flash T5 realization!☆60Updated 7 months ago
- ☆47Updated 4 months ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆56Updated 2 years ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"☆58Updated 3 months ago
- Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"☆21Updated 4 months ago
- [arXiv preprint] Official Repository for "Evaluating Language Models as Synthetic Data Generators"☆30Updated last month
- Code for Zero-Shot Tokenizer Transfer☆119Updated this week
- Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval☆39Updated 2 months ago
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆127Updated last year
- The GitHub repo for Goal Driven Discovery of Distributional Differences via Language Descriptions☆68Updated last year
- Experiments with generating opensource language model assistants☆97Updated last year
- Advanced Reasoning Benchmark Dataset for LLMs☆45Updated last year
- [ACL 2023] Gradient Ascent Post-training Enhances Language Model Generalization☆29Updated 4 months ago