Lightning-AI / forked-pdbLinks
Python pdb for multiple processes
☆51Updated 2 months ago
Alternatives and similar repositories for forked-pdb
Users that are interested in forked-pdb are comparing it to the libraries listed below
Sorting:
- Odysseus: Playground of LLM Sequence Parallelism☆72Updated last year
- ☆41Updated 4 years ago
- Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"☆119Updated last year
- Triton implementation of FlashAttention2 that adds Custom Masks.☆128Updated 11 months ago
- ☆29Updated 2 years ago
- Torch Distributed Experimental☆117Updated last year
- Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large …☆65Updated 3 years ago
- ☆106Updated last year
- pytorch-profiler☆51Updated 2 years ago
- ☆32Updated last year
- [KDD'22] Learned Token Pruning for Transformers☆98Updated 2 years ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆152Updated last month
- Megatron's multi-modal data loader☆232Updated last week
- [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling☆86Updated 2 years ago
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆66Updated 7 months ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated 2 years ago
- See details in https://github.com/pytorch/xla/blob/r1.12/torch_xla/distributed/fsdp/README.md☆24Updated 2 years ago
- ☆11Updated last year
- ☆55Updated last year
- ☆123Updated 2 months ago
- ☆107Updated 11 months ago
- Examples for MS-AMP package.☆29Updated 3 weeks ago
- Sparse Backpropagation for Mixture-of-Expert Training☆30Updated last year
- ☆147Updated 2 years ago
- Implementation of a Transformer, but completely in Triton☆273Updated 3 years ago
- ☆210Updated 2 years ago
- ☆22Updated last year
- Profile the GPU memory usage of every line in a Pytorch code☆83Updated 7 years ago
- ☆22Updated 3 months ago
- Linear Attention Sequence Parallelism (LASP)☆85Updated last year