Alternatives and similar repositories for megfile:

Users that are interested in megfile are comparing it to the libraries listed below

• HFAiLab / ffrecord
  FireFlyer Record file format, writer and reader for DL training samples.
  ☆139Updated 2 years ago
• megvii-research / hpman
  A hyperparameter manager for deep learning experiments.
  ☆96Updated 2 years ago
• YellowOldOdd / SDBI
  Simple Dynamic Batching Inference
  ☆145Updated 2 years ago
• TencentARC / mllm-npu
  mllm-npu: training multimodal large language models on Ascend NPUs
  ☆90Updated 5 months ago
• MegEngine / mgeconvert
  MegEngine到其他框架的转换器
  ☆69Updated last year
• mit-han-lab / patch_conv
  Patch convolution to avoid large GPU memory usage of Conv2D
  ☆85Updated 3 weeks ago
• ModelTC / NART
  NART = NART is not A RunTime, a deep learning inference framework.
  ☆38Updated last year
• Oneflow-Inc / vision
  Datasets, Transforms and Models specific to Computer Vision
  ☆84Updated last year
• DeepLink-org / CVFusion
  CVFusion is an open-source deep learning compiler to fuse the OpenCV operators.
  ☆29Updated 2 years ago
• Ascend / AscendSpeed
  ☆76Updated last year
• octoml / deformable-attention-kernel
  TVMScript kernel for deformable attention
  ☆24Updated 3 years ago
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆338Updated last week
• Oneflow-Inc / models
  Models and examples built with OneFlow
  ☆96Updated 4 months ago
• ppwwyyxx / RAM-multiprocess-dataloader
  Demystify RAM Usage in Multi-Process Data Loaders
  ☆187Updated last year
• FlagOpen / FlagAttention
  A collection of memory efficient attention operators implemented in the Triton language.
  ☆240Updated 8 months ago
• Ascend / pytorch
  Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch
  ☆303Updated this week
• MegEngine / MegRay
  A communication library for deep learning
  ☆50Updated 6 months ago
• volcengine / veScale
  A PyTorch Native LLM Training Framework
  ☆732Updated last month
• Oneflow-Inc / libai
  LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
  ☆398Updated last month
• alibaba / EasyParallelLibrary
  Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
  ☆266Updated last year
• bytedance / ByteTransformer
  optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
  ☆469Updated 11 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆428Updated this week
• FateScript / dotfiles
  useful dotfiles included vim, zsh, tmux and vscode
  ☆18Updated last month
• kaiyuyue / torchshard
  Slicing a PyTorch Tensor Into Parallel Shards
  ☆298Updated 3 years ago
• ucbrise / actnn
  ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
  ☆201Updated 2 years ago
• megvii-research / Sparsebit
  A model compression and acceleration toolbox based on pytorch.
  ☆329Updated last year
• bytedance / matxscript
  A high-performance, extensible Python AOT compiler.
  ☆416Updated last year
• xdit-project / DistVAE
  A parallelism VAE avoids OOM for high resolution image generation
  ☆53Updated 3 weeks ago
• silverbulletmdc / showdata
  Large scale image dataset visiualization tool.
  ☆119Updated last year
• itsliupeng / torchnvjpeg
  Decode JPEG image on GPU using PyTorch
  ☆86Updated last year