A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to copy code and launch discussions about the problems you have encoured.
☆57Jul 4, 2023Updated 3 years ago
Alternatives and similar repositories for llama-pipeline-parallel
Users that are interested in llama-pipeline-parallel are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆97Feb 5, 2024Updated 2 years ago
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆224Nov 21, 2023Updated 2 years ago
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆84Jan 14, 2025Updated last year
- ☆15Dec 28, 2022Updated 3 years ago
- 微调阿里开源的文字检测模型，利用合合识别返回的OCR结果作为初始训练数据，对模型进行优化训练，使其更加适应1万张图片的具体场景，提高文字识别的精度。☆10Dec 9, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Simhash and near-duplicate detection☆17Dec 6, 2013Updated 12 years ago
- code for COLING paper "A Hybrid Model of Classification and Generation for Spatial Relation Extraction"☆10Oct 20, 2022Updated 3 years ago
- Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning☆12Aug 23, 2025Updated last year
- ☆11Oct 8, 2023Updated 2 years ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆42Jan 4, 2024Updated 2 years ago
- Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning☆20Feb 4, 2022Updated 4 years ago
- Official implementation of the paper "From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large L…☆55Jun 24, 2024Updated 2 years ago
- Fully open reproduction of DeepSeek-R1☆11Mar 24, 2025Updated last year
- 集中管理所有的prompt。☆14Nov 27, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Customized Inference Engine for Multiverse Models☆26Jun 27, 2025Updated last year
- [ACL'25] Official code of curiosity-driven RLHF☆16Jun 22, 2025Updated last year
- An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).☆61Aug 13, 2024Updated 2 years ago
- ☆12Aug 15, 2023Updated 3 years ago
- ☆17Oct 15, 2023Updated 2 years ago
- ☆28Dec 2, 2024Updated last year
- ☆14Jul 13, 2022Updated 4 years ago
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆761Sep 27, 2024Updated last year
- Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"☆16Nov 11, 2024Updated last year
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- [IJCAI'24] Official code for our paper "Make Graph Neural Networks Great Again: A Generic Integration Paradigm of Topology-Free Patterns …☆15Jul 3, 2025Updated last year
- Source code for ICLR 2021 paper : Pre-training Text-to-Text Transformers for Concept-Centric Common Sense☆25Sep 16, 2021Updated 4 years ago
- ☆16May 15, 2025Updated last year
- ☆19Nov 21, 2024Updated last year
- ☆16Apr 11, 2022Updated 4 years ago
- ☆20Mar 18, 2026Updated 5 months ago
- Towards Systematic Measurement for Long Text Quality☆39Sep 5, 2024Updated last year
- Ring attention implementation with flash attention☆1,049Sep 10, 2025Updated 11 months ago
- This is the open source version of HPL-MXP. The code performance has been verified on Frontier☆18Jul 9, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Visual and Embodied Concepts evaluation benchmark☆21Oct 10, 2023Updated 2 years ago
- 高性能文本 Tokenizer 库☆32Feb 2, 2024Updated 2 years ago
- The Web Conference 2020: Structure-Feature based Graph Self-adaptive Pooling☆30Apr 21, 2020Updated 6 years ago
- ☆16May 28, 2021Updated 5 years ago
- ☆20Nov 5, 2024Updated last year
- ☆21Mar 7, 2024Updated 2 years ago
- Code of Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning☆15Apr 29, 2024Updated 2 years ago