some mixture of experts architecture implementations
☆27Mar 22, 2024Updated 2 years ago
Alternatives and similar repositories for MoE
Users that are interested in MoE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- nanoGPT-like codebase for LLM training☆118Nov 7, 2025Updated 6 months ago
- Optimize GEMM with tensorcore step by step☆37Dec 17, 2023Updated 2 years ago
- Implementation of "Towards Understanding Mixture of Experts in Deep Learning", NeurIPS 2022☆10Jan 6, 2023Updated 3 years ago
- Reimplementation of https://github.com/montemac/algebraic_value_editing in pure PyTorch for efficiency on large models☆11Jun 28, 2023Updated 2 years ago
- ☆12Mar 16, 2022Updated 4 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆27Apr 23, 2026Updated last month
- An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.☆21Nov 28, 2022Updated 3 years ago
- PyTorch Code for the Paper: "Exploiting Uncertainty of Loss Landscape for Stochastic Optimization [Bhaskara et al. (2019)]☆16Apr 30, 2026Updated 3 weeks ago
- ☆16May 14, 2025Updated last year
- ☆19Jun 10, 2024Updated last year
- An extention to the GaLore paper, to perform Natural Gradient Descent in low rank subspace☆19Oct 21, 2024Updated last year
- [ICLRW'26] EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation☆45Apr 21, 2026Updated last month
- Federated Learning - PyTorch☆15Jun 27, 2021Updated 4 years ago
- A Rust library for creating solvers in the OP Stack's dispute protocol☆19Jan 15, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- An recognition oriented deep learning framework for biometric sample quality assessment☆12Aug 24, 2023Updated 2 years ago
- Virtual Adversarial Training (VAT) techniques in PyTorch☆17Jul 19, 2022Updated 3 years ago
- Dynamic Telegram Trading Bot☆20Feb 21, 2025Updated last year
- General Matrix Multiplication using NVIDIA Tensor Cores☆28Jan 25, 2025Updated last year
- Jax implementation of the AdaHessian optimizer☆19Mar 11, 2021Updated 5 years ago
- Starlight: A Kernel Optimizer for GPU Processing☆16Jan 10, 2024Updated 2 years ago
- Minimal implementation of the Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models paper (ArXiv 20232401.01335)☆29Mar 1, 2024Updated 2 years ago
- A collection of different ways to implement accessing and modifying internal model activations for LLMs☆24Oct 18, 2024Updated last year
- Compression schema for gradients of activations in backward pass☆45Jul 26, 2023Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Pytorch implementation of HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models☆28Mar 22, 2024Updated 2 years ago
- ☆25Mar 15, 2023Updated 3 years ago
- Towards Understanding the Mixture-of-Experts Layer in Deep Learning☆35Dec 12, 2023Updated 2 years ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs with LoRA support.☆30Feb 19, 2024Updated 2 years ago
- Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"☆51Jun 30, 2025Updated 10 months ago
- Pytorch implementation of "Oscillation-Reduced MXFP4 Training for Vision Transformers" on DeiT Model Pre-training☆40May 4, 2026Updated 3 weeks ago
- The official repo of continuous speculative decoding☆34Mar 28, 2025Updated last year
- Python library containing BART query generation and BERT-based Siamese models for neural retrieval.☆40Oct 30, 2020Updated 5 years ago
- Conformal Language Modeling☆31Dec 21, 2023Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆252Jun 15, 2025Updated 11 months ago
- No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models (ICLR 2022)☆29Feb 9, 2022Updated 4 years ago
- A framework for implementing equivariant DL☆10May 25, 2021Updated 5 years ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆101Sep 30, 2024Updated last year
- Automatically take good care of your preemptible TPUs☆37May 15, 2023Updated 3 years ago
- ☆50Jan 18, 2024Updated 2 years ago
- Various transformers for FSDP research☆38Nov 11, 2022Updated 3 years ago