huggingface / distill-bloom-deepspeedLinks
Teacher - student distillation using DeepSpeed
☆19Updated 3 years ago
Alternatives and similar repositories for distill-bloom-deepspeed
Users that are interested in distill-bloom-deepspeed are comparing it to the libraries listed below
Sorting:
- ☆127Updated last year
- Transformers at any scale☆41Updated last year
- [ACL 2023] Gradient Ascent Post-training Enhances Language Model Generalization☆28Updated last year
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆43Updated 2 weeks ago
- [ICLR 2023] Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners☆116Updated 3 months ago
- Adding new tasks to T0 without catastrophic forgetting☆33Updated 2 years ago
- [AAAI 2024] Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following☆78Updated last year
- [ICML 2023] Exploring the Benefits of Training Expert Language Models over Instruction Tuning☆99Updated 2 years ago
- SILO Language Models code repository☆83Updated last year
- Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆39Updated 11 months ago
- ☆26Updated last year
- [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling☆39Updated last year
- ☆76Updated last year
- IntructIR, a novel benchmark specifically designed to evaluate the instruction following ability in information retrieval models. Our foc…☆32Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- ☆22Updated 2 months ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆96Updated 2 years ago
- ☆95Updated last year
- CodeUltraFeedback: aligning large language models to coding preferences (TOSEM 2025)☆71Updated last year
- [NeurIPS'23] Speculative Decoding with Big Little Decoder☆94Updated last year
- Code repository for the c-BTM paper☆107Updated 2 years ago
- A scalable automated alignment method for large language models. Resources for "Aligning Large Language Models via Self-Steering Optimiza…☆20Updated 10 months ago
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆135Updated 2 years ago
- [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models☆55Updated 8 months ago
- Repository for the "Understanding and Mitigating Language Confusion in LLMs" paper☆29Updated last year
- Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)☆61Updated last year
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆138Updated 2 years ago
- A repository for research on medium sized language models.☆78Updated last year
- Sparse Backpropagation for Mixture-of-Expert Training☆29Updated last year
- ☆19Updated 2 years ago