NVIDIA / cuda-checkpoint
CUDA checkpoint and restore utility
☆193Updated 5 months ago
Related projects: ⓘ
- Efficient serverless deployment for large AI models.☆178Updated this week
- A library to analyze PyTorch traces.☆270Updated last week
- Module, Model, and Tensor Serialization/Deserialization☆175Updated 3 weeks ago
- MSCCL++: A GPU-driven communication stack for scalable AI applications☆233Updated this week
- A low-latency & high-throughput serving engine for LLMs☆174Updated last week
- NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.☆108Updated 10 months ago
- NCCL Profiling Kit☆104Updated 2 months ago
- ☆225Updated 3 weeks ago
- NVIDIA NCCL Tests for Distributed Training☆59Updated last month
- Cloud Native Benchmarking of Foundation Models☆13Updated 2 months ago
- Efficient and easy multi-instance LLM serving☆119Updated this week
- This is a plugin which lets EC2 developers use libfabric as network provider while running NCCL applications.☆138Updated this week
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆92Updated 6 months ago
- PArametrized Recommendation and Ai Model benchmark is a repository for development of numerous uBenchmarks as well as end to end nets for…☆118Updated last week
- A resilient distributed training framework☆78Updated 5 months ago
- An interference-aware scheduler for fine-grained GPU sharing☆92Updated 4 months ago
- Dynolog is a telemetry daemon for performance monitoring and tracing. It exports metrics from different components in the system like the…☆258Updated this week
- cricket is a virtualization solution for GPUs☆139Updated 8 months ago
- ☆32Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆95Updated this week
- ☆42Updated this week
- Applied AI experiments and examples for PyTorch☆123Updated last month
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆194Updated last week
- ☆35Updated 2 months ago
- Deep Learning Energy Measurement and Optimization☆193Updated last week
- MIG Partition Editor for NVIDIA GPUs☆163Updated this week
- RDMA and SHARP plugins for nccl library☆154Updated this week
- Splits single Nvidia GPU into multiple partitions with complete compute and memory isolation (wrt to performace) between the partitions☆152Updated 5 years ago
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆145Updated this week
- Dynamic Memory Management for Serving LLMs without PagedAttention☆186Updated last month