Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
β2,555Jan 11, 2026Updated 10 months ago
Alternatives and similar repositories for hivemind
Users that are interested in hivemind are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- πΈ Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloadingβ10,716Sep 7, 2024Updated 2 years ago
- Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"β151Dec 11, 2023Updated 2 years ago
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)β119Jan 13, 2022Updated 4 years ago
- "Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts" (NeurIPS 2020), original PyTorch implemenβ¦β56Nov 5, 2020Updated 6 years ago
- Training a model similar to OpenAI DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)β27May 29, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways β’ AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Memory-efficient transformer. Work in progress.β19Sep 17, 2022Updated 4 years ago
- prime is a framework for efficient, globally distributed training of AI models over the internet.β857Jul 17, 2026Updated 4 months ago
- OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Trainingβ609Jul 17, 2026Updated 4 months ago
- PyTorch extensions for high performance and large scale training.β3,404Apr 26, 2025Updated last year
- Supercharge Your Model Trainingβ5,503Apr 29, 2026Updated 6 months ago
- Accessible large language models via k-bit quantization for PyTorch.β8,543Nov 2, 2026Updated 2 weeks ago
- An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed librariesβ7,476Nov 7, 2026Updated last week
- Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.β31,450Updated this week
- Efficient Deep Learning Systems course materialsβ1,053May 28, 2026Updated 5 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer β’ AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- "Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices", official implementationβ30Feb 4, 2025Updated last year
- π A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (iβ¦β9,951Updated this week
- The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build cuβ¦β10,852Updated this week
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.β43,333Updated this week
- Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)β9,638Aug 26, 2026Updated 2 months ago
- RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable)β¦β14,783Nov 4, 2026Updated 2 weeks ago
- A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)β4,758Jan 8, 2024Updated 2 years ago
- Hackable and optimized Transformers building blocks, supporting a composable construction.β10,582Updated this week
- A Python-level JIT compiler designed to make unmodified PyTorch programs faster.β1,079Apr 17, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean β’ AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and moreβ36,644Updated this week
- a libp2p-backed daemon wrapping the functionalities of go-libp2p for use in other languagesβ12Feb 9, 2025Updated last year
- Development repository for the Triton language and compilerβ20,600Updated this week
- functorch is JAX-like composable function transforms for PyTorch.β1,434Aug 21, 2025Updated last year
- Go ahead and axolotl questionsβ12,738Updated this week
- Library for 8-bit optimizers and quantization routines.β778Aug 18, 2022Updated 4 years ago
- Running large language models on a single GPU for throughput-oriented scenarios.β9,336Oct 28, 2024Updated 2 years ago
- Accelerated deep learning R&Dβ3,389Jul 8, 2026Updated 4 months ago
- AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (Nβ¦β4,733Oct 7, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean β’ AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- You like pytorch? You like micrograd? You love tinygrad! β€οΈβ33,942Updated this week
- Training and serving large-scale neural networks with auto parallelization.β3,176Dec 9, 2023Updated 2 years ago
- Tools for merging pretrained large language models.β7,475Oct 31, 2026Updated 2 weeks ago
- Fast and memory-efficient exact attentionβ25,316Updated this week
- A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) trainingβ25,036Aug 15, 2024Updated 2 years ago
- A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Autoβ¦β18,765Updated this week
- QLoRA: Efficient Finetuning of Quantized LLMsβ11,072Jun 10, 2024Updated 2 years ago
Are these results useful?