RulinShao / FastCkptLinks
Python package for rematerialization-aware gradient checkpointing
☆25Updated last year
Alternatives and similar repositories for FastCkpt
Users that are interested in FastCkpt are comparing it to the libraries listed below
Sorting:
- ☆75Updated 4 years ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆214Updated last year
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆40Updated 2 years ago
- ☆94Updated 3 years ago
- PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".☆90Updated 2 years ago
- ☆85Updated 3 years ago
- ☆110Updated last year
- ☆28Updated 4 years ago
- PyTorch bindings for CUTLASS grouped GEMM.☆110Updated 3 months ago
- A resilient distributed training framework☆94Updated last year
- ☆42Updated 2 years ago
- nnScaler: Compiling DNN models for Parallel Training☆115Updated last week
- ☆40Updated last year
- Best practices for training DeepSeek, Mixtral, Qwen and other MoE models using Megatron Core.☆64Updated this week
- Bridge Megatron-Core to Hugging Face/Reinforcement Learning☆97Updated this week
- Utility scripts for PyTorch (e.g. Memory profiler that understands more low-level allocations such as NCCL)☆49Updated 3 weeks ago
- FTPipe and related pipeline model parallelism research.☆41Updated 2 years ago
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆217Updated last year
- Supplemental materials for The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning☆23Updated 3 months ago
- ☆69Updated last year
- A minimal implementation of vllm.☆51Updated last year
- Triton-based implementation of Sparse Mixture of Experts.☆233Updated this week
- PyTorch bindings for CUTLASS grouped GEMM.☆137Updated last month
- ☆56Updated 8 months ago
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention☆47Updated 3 weeks ago
- Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.☆66Updated 5 months ago
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆58Updated 9 months ago
- TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators☆74Updated 2 months ago
- ☆81Updated 3 months ago
- ☆114Updated 9 months ago