18907305772 / FuseAI
FuseAI Project
☆83Updated last month
Alternatives and similar repositories for FuseAI:
Users that are interested in FuseAI are comparing it to the libraries listed below
- Reformatted Alignment☆114Updated 5 months ago
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆140Updated 6 months ago
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆75Updated last year
- Unofficial implementation of AlpaGasus☆90Updated last year
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper☆29Updated 9 months ago
- ☆48Updated last year
- ☆76Updated 2 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆76Updated last year
- ☆101Updated 3 months ago
- Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper☆128Updated 7 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆129Updated 9 months ago
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆136Updated 4 months ago
- Implementations of online merging optimizers proposed by Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment☆74Updated 9 months ago
- A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.☆84Updated last year
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆205Updated 9 months ago
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆241Updated 3 months ago
- ☆92Updated 3 months ago
- Hammer: Robust Function-Calling for On-Device Language Models via Function Masking☆63Updated last month
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆121Updated 2 months ago
- Co-LLM: Learning to Decode Collaboratively with Multiple Language Models☆109Updated 10 months ago
- The official repo for "LLoCo: Learning Long Contexts Offline"☆115Updated 9 months ago
- This is the official repository for Inheritune.☆109Updated last month
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆44Updated 8 months ago
- Code implementation of synthetic continued pretraining☆94Updated 2 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆62Updated last year
- ☆98Updated 5 months ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆36Updated last year