sanyalsunny111 / Early_Weight_AvgLinks
[COLM 2024] Early Weight Averaging meets High Learning Rates for LLM Pre-training
☆16Updated 9 months ago
Alternatives and similar repositories for Early_Weight_Avg
Users that are interested in Early_Weight_Avg are comparing it to the libraries listed below
Sorting:
- In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning☆35Updated last year
- Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch☆30Updated 2 weeks ago
- ☆64Updated last year
- MEXMA: Token-level objectives improve sentence representations☆41Updated 6 months ago
- official repo of AAAI2024 paper Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularization☆13Updated last year
- IntructIR, a novel benchmark specifically designed to evaluate the instruction following ability in information retrieval models. Our foc…