Alternatives and similar repositories for megfile

Users that are interested in megfile are comparing it to the libraries listed below

• FateScript / dotfiles
  useful dotfiles included vim, zsh, tmux and vscode
  ☆19Updated 2 weeks ago
• HFAiLab / ffrecord
  FireFlyer Record file format, writer and reader for DL training samples.
  ☆238Updated 3 years ago
• Tele-AI / TeleTron
  To pioneer training long-context multi-modal transformer models
  ☆68Updated 5 months ago
• silverbulletmdc / showdata
  Large scale image dataset visiualization tool.
  ☆121Updated 2 months ago
• ppwwyyxx / RAM-multiprocess-dataloader
  Demystify RAM Usage in Multi-Process Data Loaders
  ☆205Updated 2 years ago
• TencentARC / mllm-npu
  mllm-npu: training multimodal large language models on Ascend NPUs
  ☆95Updated last year
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆58Updated 5 months ago
• SandAI-org / MagiAttention
  A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training
  ☆626Updated this week
• stepfun-ai / Step3
  ☆449Updated 5 months ago
• Jittor / JittorInfer
  JittorInfer is a high-performance C++ inference framework designed for large language models on Huawei's Ascend AI processor.
  ☆78Updated last week
• Ascend / pytorch
  Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch
  ☆483Updated this week
• MegEngine / mgeconvert
  MegEngine到其他框架的转换器
  ☆70Updated 2 years ago
• octoml / deformable-attention-kernel
  TVMScript kernel for deformable attention
  ☆25Updated 4 years ago
• open-mmlab / ecosystem
  ☆36Updated 2 years ago
• Oneflow-Inc / vision
  Datasets, Transforms and Models specific to Computer Vision
  ☆90Updated 2 years ago
• ByteDance-Seed / VeOmni
  VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
  ☆1,576Updated last week
• ICT-ANS / StarLight
  ☆61Updated last year
• YellowOldOdd / SDBI
  Simple Dynamic Batching Inference
  ☆145Updated 3 years ago
• itsliupeng / torchnvjpeg
  Decode JPEG image on GPU using PyTorch
  ☆93Updated 2 years ago
• open-mmlab / mmeval
  A unified evaluation library for multiple machine learning libraries
  ☆269Updated last year
• RiseAI-Sys / DAX
  High performance inference engine for diffusion models
  ☆103Updated 4 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆637Updated 2 weeks ago
• NVIDIA / Megatron-Energon
  Megatron's multi-modal data loader
  ☆310Updated this week
• luo3300612 / grid_sample1d
  pytorch cuda extension of grid_sample1d
  ☆49Updated 4 years ago
• mit-han-lab / patch_conv
  Patch convolution to avoid large GPU memory usage of Conv2D
  ☆95Updated last year
• DeepLink-org / CVFusion
  CVFusion is an open-source deep learning compiler to fuse the OpenCV operators.
  ☆33Updated 3 years ago
• megvii-research / Sparsebit
  A model compression and acceleration toolbox based on pytorch.
  ☆333Updated 2 years ago
• Oneflow-Inc / libai
  LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
  ☆405Updated 6 months ago
• ByteDance-Seed / ByteCheckpoint
  ByteCheckpoint: An Unified Checkpointing Library for LFMs
  ☆268Updated last month
• mini-sora / MiniSora-DiT
  minisora-DiT, a DiT reproduction based on XTuner from the open source community MiniSora
  ☆40Updated last year