foundation-model-stack / fms-fsdp

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.
190Updated 2 weeks ago

Related projects

Alternatives and complementary repositories for fms-fsdp