Alternatives and similar repositories for fms-fsdp:

Users that are interested in fms-fsdp are comparing it to the libraries listed below

• foundation-model-stack / foundation-model-stack
  🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.
  ☆192Updated this week
• pytorch-labs / float8_experimental
  This repository contains the experimental PyTorch native float8 training UX
  ☆222Updated 8 months ago
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆256Updated 3 weeks ago
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆211Updated 4 months ago
• lucidrains / ring-attention-pytorch
  Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
  ☆510Updated 5 months ago
• gpu-mode / ring-attention
  ring-attention experiments
  ☆129Updated 5 months ago
• haoliuhl / ringattention
  Large Context Attention
  ☆700Updated 2 months ago
• huggingface / picotron_tutorial
  ☆166Updated 2 months ago
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆285Updated this week
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆207Updated 7 months ago
• gpu-mode / triton-index
  Cataloging released Triton kernels.
  ☆213Updated 3 months ago
• lucidrains / speculative-decoding
  Explorations into some recent techniques surrounding speculative decoding
  ☆253Updated 3 months ago
• NVIDIA / kvpress
  LLM KV cache compression made easy
  ☆452Updated 3 weeks ago
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆734Updated this week
• Azure / MS-AMP
  Microsoft Automatic Mixed Precision Library
  ☆590Updated 6 months ago
• Deep-Learning-Profiling-Tools / triton-viz
  ☆198Updated this week
• zinccat / Awesome-Triton-Kernels
  Collection of kernels written in Triton language
  ☆118Updated last week
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆470Updated this week
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆381Updated this week
• huggingface / kernels
  Load compute kernels from the Hub
  ☆113Updated this week
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆97Updated this week
• neuralmagic / AutoFP8
  ☆185Updated 6 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆112Updated 4 months ago
• stanford-futuredata / stk
  ☆103Updated 7 months ago
• MekkCyber / CutlassAcademy
  A curated collection of resources, tutorials, and best practices for learning and mastering NVIDIA CUTLASS
  ☆156Updated 3 weeks ago
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆366Updated last month
• Infini-AI-Lab / Sequoia
  scalable and robust tree-based speculative decoding algorithm
  ☆341Updated 2 months ago
• HazyResearch / lolcats
  Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"
  ☆230Updated 2 months ago
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆98Updated this week
• ScalingIntelligence / KernelBench
  KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems
  ☆263Updated this week