A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to copy code and launch discussions about the problems you have encoured.
☆57Jul 4, 2023Updated 3 years ago
Alternatives and similar repositories for llama-pipeline-parallel
Users that are interested in llama-pipeline-parallel are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆98Feb 5, 2024Updated 2 years ago
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆224Nov 21, 2023Updated 2 years ago
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆84Jan 14, 2025Updated last year
- [ICLR 2022] Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators☆27Jul 26, 2023Updated 3 years ago
- [COLING2022] Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection☆10Aug 3, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- The official implementation for the paper "ST-MTM: Masked Time Series Modeling with Seasonal-Trend Decomposition for Time Series Forecast…☆12May 22, 2025Updated last year
- code for COLING paper "A Hybrid Model of Classification and Generation for Spatial Relation Extraction"☆10Oct 20, 2022Updated 4 years ago
- ☆15Dec 28, 2022Updated 3 years ago
- Pipeline Parallelism for PyTorch☆788Aug 21, 2024Updated 2 years ago
- Mosaic Representation Learning for Self-supervised Visual Pre-training (ICLR2023, Spotlight)☆15Apr 7, 2023Updated 3 years ago
- Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning☆12Aug 23, 2025Updated last year
- Unsupervised Cross-lingual Sentiment Analysis (CoNLL 2019)☆10Nov 4, 2019Updated 7 years ago
- 🖖 图谱式笔记系统,旨在提高个人笔记的使用率!☆11Jan 17, 2021Updated 5 years ago
- Customized Inference Engine for Multiverse Models☆26Jun 27, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆11Oct 8, 2023Updated 3 years ago
- ICML'25: The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models☆11May 19, 2025Updated last year
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆42Jan 4, 2024Updated 2 years ago
- LLM Training code from our paper **Chart Specification: Structural Representations for Incentivizing VLM Reasoning in Chart-to-Code Gener…☆14Aug 7, 2026Updated 3 months ago
- ☆10May 9, 2018Updated 8 years ago
- ☆12Apr 19, 2024Updated 2 years ago
- An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).☆60Aug 13, 2024Updated 2 years ago
- ☆29Dec 2, 2024Updated last year
- Repo for the EMNLP2021 paper: Lifelong Event Detection with Knowledge Transfer☆14Sep 2, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆14Jul 13, 2022Updated 4 years ago
- ☆17Feb 20, 2023Updated 3 years ago
- simple implementation of Expected Gradients and Integrated Gradients by pytorch☆13May 11, 2022Updated 4 years ago
- ☆18Aug 19, 2024Updated 2 years ago
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆759Sep 27, 2024Updated 2 years ago
- Repository for paper ``IG2: Integrated Gradient on Iterative Gradient Path for Feature Attribution‘’☆14Feb 28, 2025Updated last year
- A benchmark to evaluate search-augmented LLMs☆16Aug 28, 2025Updated last year
- Mamba SSM architecture for packed data☆10Jul 1, 2024Updated 2 years ago
- Enhaced version of Wikiextrator: A wikipedia dumps extractor☆31Oct 10, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆64Jul 30, 2023Updated 3 years ago
- Code publication to the paper "Normalized Attention Without Probability Cage"☆17Aug 24, 2026Updated 2 months ago
- ☆11Dec 1, 2020Updated 5 years ago
- Completing the Puzzle of All-in-One Event Understanding Benchmark with Event Arguments☆16Mar 12, 2024Updated 2 years ago
- ☆16Apr 11, 2022Updated 4 years ago
- This reporsitory is for the code of the paper Auxiliary Learning with Joint Task and Data Scheduling☆10Aug 27, 2022Updated 4 years ago
- superpixel average pooling, superspixel max pooling, pytorch implementations☆25Nov 14, 2018Updated 8 years ago
Are these results useful?