Sequence-level 1F1B schedule for LLMs.
☆37Aug 26, 2025Updated last year
Alternatives and similar repositories for Seq1F1B
Users that are interested in Seq1F1B are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Sequence-level 1F1B schedule for LLMs.☆19Jun 4, 2024Updated 2 years ago
- LLM training technologies developed by kwai☆74Jun 30, 2026Updated 4 months ago
- Distributed IO-aware Attention algorithm☆24Sep 24, 2025Updated last year
- ☆14Sep 22, 2025Updated last year
- Reduce kernel based on CUTLASS CuTe and TMA.☆12Sep 25, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Official repository for the paper DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines☆19Dec 8, 2023Updated 2 years ago
- Zero Bubble Pipeline Parallelism☆467May 7, 2025Updated last year
- Vocabulary Parallelism☆26Mar 10, 2025Updated last year
- Code repository for the SOSP'25 paper DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism.☆23Nov 28, 2025Updated 11 months ago
- Utility scripts for PyTorch (e.g. Make Perfetto show some disappearing kernels, Memory profiler that understands more low-level allocatio…☆115Sep 11, 2025Updated last year
- Optimize SGEMM step by step☆10Sep 4, 2024Updated 2 years ago
- [ACL 2025 main] FR-Spec: Frequency-Ranked Speculative Sampling☆56Jul 15, 2025Updated last year
- Ongoing research training transformer models at scale☆19Updated this week
- ☆16Jan 14, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Allow torch tensor memory to be released and resumed later☆282Sep 29, 2026Updated last month
- Distributed MoE in a Single Kernel [NeurIPS '25]☆332May 5, 2026Updated 6 months ago
- Efficient Long-context Language Model Training by Core Attention Disaggregation☆106Apr 7, 2026Updated 6 months ago
- Source codes for paper "BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity".☆19Jan 10, 2026Updated 9 months ago
- ☆44Sep 8, 2025Updated last year
- A benchmark suited especially for deep learning operators☆43Feb 13, 2023Updated 3 years ago
- Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.☆206May 29, 2026Updated 5 months ago
- ☆61Dec 19, 2025Updated 10 months ago
- A NCCL extension library, designed to efficiently offload GPU memory allocated by the NCCL communication library.☆123Dec 17, 2025Updated 10 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆18Sep 27, 2026Updated last month
- A shared mailbox for people and their coding agents: hand results, files and context to someone else's agent (Codex, Claude Code, opencod…☆25Updated this week
- My attempt to improve the speed of the newton schulz algorithm, starting from the dion implementation.☆42Apr 30, 2026Updated 6 months ago
- ☆27Dec 5, 2022Updated 3 years ago
- Unified tracing profiler and visualizer, one timeline from CPU to GPU/HES to in-kernel zones☆152Sep 21, 2026Updated last month
- ☆14Jun 25, 2025Updated last year
- CUDA SGEMM optimization note☆16Oct 31, 2023Updated 3 years ago
- Ring attention implementation with flash attention☆1,067Sep 10, 2025Updated last year
- ☆12Nov 21, 2025Updated 11 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Dynamic resources changes for multi-dimensional parallelism training☆31Aug 22, 2025Updated last year
- [MLSys'23] Exploiting Hardware Utilization and Adaptive Dataflow for Efficient Sparse Convolution in 3D Point Clouds.☆13Jun 22, 2023Updated 3 years ago
- PyTorch bindings for CUTLASS grouped GEMM.☆193Apr 8, 2026Updated 6 months ago
- Artifact Evaluation for SOSP 2025☆23Aug 16, 2025Updated last year
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆117Mar 20, 2025Updated last year
- ☆10Nov 9, 2023Updated 2 years ago
- A lightweight design for computation-communication overlap.☆252Jan 20, 2026Updated 9 months ago