cisco-open / pymultiworldLinks
A framework for PyTorch to enable fault management for collective communication libraries (CCL) such as NCCL
☆20Updated last week
Alternatives and similar repositories for pymultiworld
Users that are interested in pymultiworld are comparing it to the libraries listed below
Sorting:
- Ship correct and fast LLM kernels to PyTorch☆127Updated last week
- Nsight Python is a Python kernel profiling interface based on NVIDIA Nsight Tools☆77Updated this week
- JAX backend for SGL☆205Updated this week
- torchcomms: a modern PyTorch communications API☆314Updated this week
- Microsoft Collective Communication Library☆66Updated last year
- Triton-based Symmetric Memory operators and examples☆70Updated 2 months ago
- Toolchain built around the Megatron-LM for Distributed Training☆81Updated 3 weeks ago
- extensible collectives library in triton☆91Updated 8 months ago
- Allow torch tensor memory to be released and resumed later☆191Updated 3 weeks ago
- Autonomous GPU Kernel Generation via Deep Agents☆192Updated last week
- A resilient distributed training framework☆96Updated last year
- A NCCL extension library, designed to efficiently offload GPU memory allocated by the NCCL communication library.☆74Updated last week
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆256Updated 2 weeks ago
- 🏙 Interactive performance profiling and debugging tool for PyTorch neural networks.☆64Updated 11 months ago
- Automated Parallelization System and Infrastructure for Multiple Ecosystems☆82Updated last year
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆45Updated last year
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆162Updated this week
- JaxPP is a library for JAX that enables flexible MPMD pipeline parallelism for large-scale LLM training☆61Updated last week
- Applied AI experiments and examples for PyTorch☆311Updated 4 months ago
- Evaluating Large Language Models for CUDA Code Generation ComputeEval is a framework designed to generate and evaluate CUDA code from Lar…☆87Updated last month
- ring-attention experiments☆160Updated last year
- Accelerating MoE with IO and Tile-aware Optimizations☆469Updated this week
- Triton-based implementation of Sparse Mixture of Experts.☆259Updated 2 months ago
- A minimal implementation of vllm.☆64Updated last year
- Efficient Long-context Language Model Training by Core Attention Disaggregation☆73Updated this week
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆43Updated 3 years ago
- ☆115Updated last year
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆204Updated last year
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆250Updated 2 weeks ago
- PyTorch bindings for CUTLASS grouped GEMM.☆135Updated 6 months ago