pytorch / torchft
PyTorch per step fault tolerance (actively under development)
☆243Updated this week
Alternatives and similar repositories for torchft:
Users that are interested in torchft are comparing it to the libraries listed below
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆221Updated last week
- This repository contains the experimental PyTorch native float8 training UX☆221Updated 6 months ago
- LLM KV cache compression made easy☆394Updated this week
- Applied AI experiments and examples for PyTorch☆224Updated this week
- ☆197Updated 3 weeks ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆514Updated this week
- Scalable and Performant Data Loading☆217Updated last week
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆172Updated last week
- Cataloging released Triton kernels.☆167Updated last month
- Fast low-bit matmul kernels in Triton☆232Updated this week
- ☆177Updated this week
- Fast Matrix Multiplications for Lookup Table-Quantized LLMs☆229Updated this week
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆187Updated this week
- ☆133Updated this week
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆154Updated 2 months ago
- ring-attention experiments☆123Updated 4 months ago
- A safetensors extension to efficiently store sparse quantized tensors on disk☆75Updated this week
- Google TPU optimizations for transformers models☆98Updated 3 weeks ago
- extensible collectives library in triton☆82Updated 4 months ago
- Minimalistic 4D-parallelism distributed training framework for education purpose☆722Updated last week
- Efficient LLM Inference over Long Sequences☆357Updated this week
- Helpful tools and examples for working with flex-attention☆631Updated this week
- A library for unit scaling in PyTorch☆122Updated 2 months ago
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆117Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆259Updated 4 months ago
- seqax = sequence modeling + JAX☆143Updated 7 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆274Updated this week
- Megatron's multi-modal data loader☆162Updated this week
- A library to analyze PyTorch traces.☆332Updated this week
- ☆284Updated this week