BryceZhuo / HybridNormLinks
The official implementation of HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
☆18Updated 10 months ago
Alternatives and similar repositories for HybridNorm
Users that are interested in HybridNorm are comparing it to the libraries listed below
Sorting:
- ☆16Updated last year
- [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623☆89Updated last year
- Code for paper "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"☆84Updated last year
- Code for paper "Patch-Level Training for Large Language Models"☆96Updated 2 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆78Updated last year
- FocusLLM: Scaling LLM’s Context by Parallel Decoding☆44Updated last year
- [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens☆25Updated 2 years ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆41Updated 3 weeks ago
- Code for paper: Long cOntext aliGnment via efficient preference Optimization☆23Updated 3 months ago
- The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"☆40Updated last year
- [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models☆57Updated 11 months ago
- ☆19Updated 10 months ago
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆109Updated 3 months ago
- Directional Preference Alignment☆58Updated last year
- [ICLR 2025] LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization☆43Updated 10 months ago
- Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"☆48Updated 2 years ago
- Official implementation of Bootstrapping Language Models via DPO Implicit Rewards☆47Updated 9 months ago
- Official Repo for the paper: VCR: Visual Caption Restoration. Check arxiv.org/pdf/2406.06462 for details.☆32Updated 10 months ago
- Large Language Models Can Self-Improve in Long-context Reasoning☆72Updated last year
- PyTorch implementation of StableMask (ICML'24)☆15Updated last year
- ☆19Updated last year
- [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…☆38Updated 2 years ago
- ☆30Updated 2 years ago
- Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context☆41Updated last year
- Exploration of automated dataset selection approaches at large scales.☆53Updated 10 months ago
- ☆11Updated last year
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆104Updated last year
- Mixture of Attention Heads☆51Updated 3 years ago
- Codebase for Instruction Following without Instruction Tuning☆36Updated last year
- ☆64Updated last year