lucidrains / CALM-pytorchLinks
Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind
☆178Updated 8 months ago
Alternatives and similar repositories for CALM-pytorch
Users that are interested in CALM-pytorch are comparing it to the libraries listed below
Sorting:
- Multipack distributed sampler for fast padding-free training of LLMs☆188Updated 9 months ago
- ☆258Updated last year
- Self-Alignment with Principle-Following Reward Models☆161Updated 3 weeks ago
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆202Updated last year
- ☆97Updated 11 months ago
- Code for Zero-Shot Tokenizer Transfer☆128Updated 4 months ago
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆170Updated 5 months ago
- ☆179Updated last year
- Scaling Data-Constrained Language Models☆334Updated 8 months ago
- Understand and test language model architectures on synthetic tasks.☆197Updated 2 months ago
- ☆131Updated 6 months ago
- Official repository of NEFTune: Noisy Embeddings Improves Instruction Finetuning☆396Updated last year
- LLM-Merging: Building LLMs Efficiently through Merging☆197Updated 8 months ago
- Language models scale reliably with over-training and on downstream tasks☆97Updated last year
- DSIR large-scale data selection framework for language model training☆249Updated last year
- ☆149Updated last year
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆123Updated last year
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆134Updated last year
- ☆219Updated 11 months ago
- This is the repo for the paper Shepherd -- A Critic for Language Model Generation☆218Updated last year
- ☆174Updated last month
- Code and Data for "Long-context LLMs Struggle with Long In-context Learning" [TMLR2025]☆106Updated 3 months ago
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆143Updated 8 months ago
- This is the official repository for Inheritune.☆111Updated 3 months ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆454Updated last year
- Simple next-token-prediction for RLHF☆226Updated last year
- ☆120Updated 8 months ago
- Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)☆190Updated last year
- X-LoRA: Mixture of LoRA Experts☆226Updated 10 months ago
- Small and Efficient Mathematical Reasoning LLMs☆71Updated last year