galatolofederico / vanilla-llama
Plain pytorch implementation of LLaMA
☆188Updated last year
Alternatives and similar repositories for vanilla-llama:
Users that are interested in vanilla-llama are comparing it to the libraries listed below
- [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆597Updated last year
- Fast Inference Solutions for BLOOM☆562Updated 5 months ago
- Official repository for LongChat and LongEval☆516Updated 10 months ago
- [NIPS2023] RRHF & Wombat☆804Updated last year
- Crosslingual Generalization through Multitask Finetuning☆529Updated 6 months ago
- ☆458Updated last year
- Official repository of NEFTune: Noisy Embeddings Improves Instruction Finetuning☆393Updated 10 months ago
- GPTQ inference Triton kernel☆298Updated last year
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆216Updated last year
- Automatically split your PyTorch models on multiple GPUs for training & inference☆650Updated last year
- LOMO: LOw-Memory Optimization☆981Updated 8 months ago
- [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization☆680Updated 7 months ago
- [COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition☆619Updated 8 months ago
- This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.☆544Updated last year
- Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch☆638Updated 3 months ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆448Updated 11 months ago
- OpenAlpaca: A Fully Open-Source Instruction-Following Model Based On OpenLLaMA☆301Updated last year
- Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets☆316Updated last year
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- Rectified Rotary Position Embeddings☆360Updated 10 months ago
- Inference code for facebook LLaMA models with Wrapyfi support☆130Updated 2 years ago
- DSIR large-scale data selection framework for language model training☆244Updated 11 months ago
- batched loras☆340Updated last year
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆308Updated 2 years ago
- llama fine-tuning with lora☆140Updated 10 months ago
- Due to restriction of LLaMA, we try to reimplement BLOOM-LoRA (much less restricted BLOOM license here https://huggingface.co/spaces/bigs…☆184Updated last year
- RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best …☆412Updated last year
- Multi-language Enhanced LLaMA☆301Updated last year
- ☆459Updated 9 months ago
- Code for fine-tuning Platypus fam LLMs using LoRA☆628Updated last year