Alternatives and similar repositories for megfile

Users that are interested in megfile are comparing it to the libraries listed below

• FateScript / dotfiles
  useful dotfiles included vim, zsh, tmux and vscode
  ☆18Updated 3 months ago
• Tele-AI / TeleTron
  To pioneer training long-context multi-modal transformer models
  ☆64Updated 4 months ago
• TencentARC / mllm-npu
  mllm-npu: training multimodal large language models on Ascend NPUs
  ☆94Updated last year
• SandAI-org / MagiAttention
  A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training
  ☆584Updated this week
• stepfun-ai / Step3
  ☆439Updated 4 months ago
• ppwwyyxx / RAM-multiprocess-dataloader
  Demystify RAM Usage in Multi-Process Data Loaders
  ☆205Updated 2 years ago
• mit-han-lab / patch_conv
  Patch convolution to avoid large GPU memory usage of Conv2D
  ☆93Updated 10 months ago
• HFAiLab / ffrecord
  FireFlyer Record file format, writer and reader for DL training samples.
  ☆237Updated 3 years ago
• octoml / deformable-attention-kernel
  TVMScript kernel for deformable attention
  ☆25Updated 4 years ago
• silverbulletmdc / showdata
  Large scale image dataset visiualization tool.
  ☆121Updated last month
• Oneflow-Inc / vision
  Datasets, Transforms and Models specific to Computer Vision
  ☆90Updated 2 years ago
• xdit-project / DistVAE
  A parallelism VAE avoids OOM for high resolution image generation
  ☆84Updated 4 months ago
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆55Updated 4 months ago
• RiseAI-Sys / DAX
  High performance inference engine for diffusion models
  ☆100Updated 3 months ago
• ByteDance-Seed / VeOmni
  VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
  ☆1,432Updated this week
• megvii-research / hpman
  A hyperparameter manager for deep learning experiments.
  ☆96Updated 3 years ago
• DeepLink-org / dlinfer
  ☆66Updated 2 weeks ago
• MegEngine / mgeconvert
  MegEngine到其他框架的转换器
  ☆69Updated 2 years ago
• OpenNLPLab / lightning-attention
  Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
  ☆338Updated 9 months ago
• NVIDIA / Megatron-Energon
  Megatron's multi-modal data loader
  ☆292Updated this week
• ICT-ANS / StarLight
  ☆61Updated last year
• NVlabs / COAT
  [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training
  ☆252Updated 4 months ago
• Tencent-Hunyuan / flex-block-attn
  flex-block-attn: an efficient block sparse attention computation library
  ☆94Updated 3 weeks ago
• thu-nics / DiTFastAttn
  ☆187Updated 11 months ago
• Oneflow-Inc / libai
  LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
  ☆406Updated 4 months ago
• Vchitect / LiteGen
  A light-weight and high-efficient training framework for accelerating diffusion tasks.
  ☆50Updated last year
• Oneflow-Inc / serving
  OneFlow Serving
  ☆20Updated 8 months ago
• mit-han-lab / Block-Sparse-Attention
  A sparse attention kernel supporting mix sparse patterns
  ☆406Updated 10 months ago
• Ascend / AscendSpeed
  ☆79Updated 2 years ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆614Updated this week