Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
☆97Feb 5, 2024Updated 2 years ago
Alternatives and similar repositories for gdGPT
Users that are interested in gdGPT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆224Nov 21, 2023Updated 2 years ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆58Jul 4, 2023Updated 2 years ago
- ☆84Sep 9, 2023Updated 2 years ago
- The official repo for our SIGIR'23 Full paper: Constructing Tree-based Index for Efficient and Effective Dense Retrieval☆28Jun 7, 2023Updated 2 years ago
- aigc evals☆10Dec 2, 2023Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆19Jan 10, 2023Updated 3 years ago
- ☆14Jul 13, 2025Updated 9 months ago
- ☆27Aug 31, 2023Updated 2 years ago
- Code for Robust Fine-tuning (RbFT)☆17Jan 31, 2025Updated last year
- Python package for rematerialization-aware gradient checkpointing☆27Oct 31, 2023Updated 2 years ago
- Code for paper titled "Towards the Law of Capacity Gap in Distilling Language Models"☆102Jul 9, 2024Updated last year
- pytorch implementation of mvp: a multi-stage vision-language pre-training framework☆11Apr 23, 2022Updated 4 years ago
- ☆43Dec 15, 2023Updated 2 years ago
- Implementation of the paper: "Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning…☆22Nov 2, 2021Updated 4 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆83Jan 14, 2025Updated last year
- Best practice for training LLaMA models in Megatron-LM☆664Jan 2, 2024Updated 2 years ago
- PyTorch bindings for CUTLASS grouped GEMM.☆186Apr 8, 2026Updated last month
- ☆24Jul 7, 2024Updated last year
- Semi-supervised spoken language understanding (SLU) via self-supervised speech and language model pretraining☆12Mar 23, 2021Updated 5 years ago
- 工业级中文语音识别系统电子书☆13Oct 30, 2020Updated 5 years ago
- [SIGIR 2025] The official repo for "Scaling Sparse and Dense Retrieval in Decoder-Only LLMs"☆20Mar 31, 2025Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,439Mar 20, 2024Updated 2 years ago
- A more efficient GLM implementation!☆54Feb 18, 2023Updated 3 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- ☆133Nov 11, 2024Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆2,246Aug 14, 2025Updated 8 months ago
- A gomoku AI based on Alpha Zero paper.☆12May 1, 2023Updated 3 years ago
- Artifact for PPoPP22 QGTC: Accelerating Quantized GNN via GPU Tensor Core.☆30Feb 12, 2022Updated 4 years ago
- KoRean based ELECTRA pre-trained models (KR-ELECTRA) for Tensorflow and PyTorch☆15Feb 13, 2022Updated 4 years ago
- ☆24Jun 24, 2020Updated 5 years ago
- Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]☆12Nov 8, 2024Updated last year
- Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“☆64Jun 5, 2024Updated last year
- [EMNLP 2025] The official implementation for paper "Agentic-R1: Distilled Dual-Strategy Reasoning"☆103Apr 21, 2026Updated 2 weeks ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Tacker: Tensor-CUDA Core Kernel Fusion for Improving the GPU Utilization while Ensuring QoS☆33Feb 10, 2025Updated last year
- NTK scaled version of ALiBi position encoding in Transformer.☆69Aug 16, 2023Updated 2 years ago
- ☆28Jul 18, 2025Updated 9 months ago
- BiLLa: A Bilingual LLaMA with Enhanced Reasoning Ability☆416Jun 1, 2023Updated 2 years ago
- Slowdown prediction module of Echo: Simulating Distributed Training at Scale☆13May 17, 2025Updated 11 months ago
- llama2 finetuning with deepspeed and lora☆176Jul 28, 2023Updated 2 years ago
- A packaged convolutional voice activity detector for noisy environments.☆14Jun 15, 2019Updated 6 years ago