Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
☆98Feb 5, 2024Updated 2 years ago
Alternatives and similar repositories for gdGPT
Users that are interested in gdGPT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆224Nov 21, 2023Updated 2 years ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆57Jul 4, 2023Updated 3 years ago
- ☆84Sep 9, 2023Updated 3 years ago
- ☆19Jan 10, 2023Updated 3 years ago
- aigc evals☆10Dec 2, 2023Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆84Jan 14, 2025Updated last year
- Best practice for training LLaMA models in Megatron-LM☆666Jan 2, 2024Updated 2 years ago
- ☆14Jul 13, 2025Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆69Jul 20, 2023Updated 3 years ago
- ☆43Dec 15, 2023Updated 2 years ago
- This is the official Python version of CoreInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Act…☆20Oct 25, 2024Updated 2 years ago
- ☆15Nov 17, 2020Updated 5 years ago
- Code for paper titled "Towards the Law of Capacity Gap in Distilling Language Models"☆102Jul 9, 2024Updated 2 years ago
- ☆27Aug 31, 2023Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A framework for editing the CoTs for better factuality☆51Dec 9, 2023Updated 2 years ago
- Implementation of the paper: "Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning…☆22Nov 2, 2021Updated 4 years ago
- Python package for rematerialization-aware gradient checkpointing☆27Oct 31, 2023Updated 2 years ago
- Code for Robust Fine-tuning (RbFT)☆19Jan 31, 2025Updated last year
- The code of Paper "Locate Then Ask: Interpretable Stepwise Reasoning for Multi-hop Question Answering".☆22Sep 1, 2022Updated 4 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆2,268Aug 14, 2025Updated last year
- Semi-supervised spoken language understanding (SLU) via self-supervised speech and language model pretraining☆12Mar 23, 2021Updated 5 years ago
- Elixir: Train a Large Language Model on a Small GPU Cluster☆15Jun 8, 2023Updated 3 years ago
- ☆10Jul 10, 2025Updated last year
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- 工业级中文语音识别系统电子书☆13Oct 30, 2020Updated 5 years ago
- ☆16Aug 26, 2026Updated 2 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆193Apr 8, 2026Updated 6 months ago
- Multi-language Enhanced LLaMA☆299Apr 13, 2023Updated 3 years ago
- Source Code for EditCoT, EMNLP 2025 Long Paper☆14Aug 21, 2025Updated last year
- Useful skills for AI agents (Claude Code, etc.)☆20Updated this week
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,448Mar 20, 2024Updated 2 years ago
- The baseline method for CCIR 22 https://www.datafountain.cn/competitions/573☆13Aug 2, 2022Updated 4 years ago
- Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]☆12Nov 8, 2024Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Our code for WSDM Cup 2023 Task 1 and 2☆10Jan 16, 2023Updated 3 years ago
- The codebase for "Group-wise Contrastive Learning for Neural Dialogue Generation" (Cai et al., Findings of EMNLP 2020)☆55Feb 24, 2021Updated 5 years ago
- ☆53Aug 14, 2024Updated 2 years ago
- ☆13Sep 8, 2021Updated 5 years ago
- ☆24Jun 24, 2020Updated 6 years ago
- Artifact for PPoPP22 QGTC: Accelerating Quantized GNN via GPU Tensor Core.☆30Feb 12, 2022Updated 4 years ago
- ☆26Jul 7, 2024Updated 2 years ago