some mixture of experts architecture implementations
☆28Mar 22, 2024Updated 2 years ago
Alternatives and similar repositories for MoE
Users that are interested in MoE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Masking Strategies for Background Bias Removal in Computer Vision Models (ICCVW OODCV 2023 paper)☆16Sep 15, 2026Updated last month
- nanoGPT-like codebase for LLM training☆120Nov 7, 2025Updated 11 months ago
- ☆17May 3, 2024Updated 2 years ago
- ☆15Apr 26, 2022Updated 4 years ago
- Code accompanying PDiscoNet: Semantically consistent part discovery for fine-grained recognition☆15Dec 10, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- High-performance GPU kernels for LLM inference in OpenAI Triton. Fused RMSNorm, SwiGLU, INT8 GEMM with benchmarks and roofline analysis.☆44Jul 22, 2026Updated 2 months ago
- ☆11Apr 13, 2023Updated 3 years ago
- Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or otherwise☆40Aug 29, 2024Updated 2 years ago
- ☆27Jan 22, 2026Updated 8 months ago
- Reimplementation of https://github.com/montemac/algebraic_value_editing in pure PyTorch for efficiency on large models☆11Jun 28, 2023Updated 3 years ago
- ☆13Dec 20, 2024Updated last year
- Learning diverse options through the Laplacian representation.☆23Jan 5, 2024Updated 2 years ago
- PyTorch Code for the Paper: "Exploiting Uncertainty of Loss Landscape for Stochastic Optimization [Bhaskara et al. (2019)]☆16Apr 30, 2026Updated 5 months ago
- A comprehensive toolkit for streamlining data editing, search, and inspection for large-scale language model training and interpretabilit…☆21Oct 30, 2025Updated 11 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆205Jan 19, 2026Updated 8 months ago
- Optimize GEMM with tensorcore step by step☆40Dec 17, 2023Updated 2 years ago
- An extention to the GaLore paper, to perform Natural Gradient Descent in low rank subspace☆19Oct 21, 2024Updated last year
- ☆20Apr 16, 2025Updated last year
- ☆19Jun 10, 2024Updated 2 years ago
- Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention (CVPR 2023)☆32Mar 28, 2023Updated 3 years ago
- ☆36May 25, 2023Updated 3 years ago
- Federated Learning - PyTorch☆15Jun 27, 2021Updated 5 years ago
- Training hybrid models for dummies.☆30Nov 1, 2025Updated 11 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A Rust library for creating solvers in the OP Stack's dispute protocol☆19Jan 15, 2024Updated 2 years ago
- Virtual Adversarial Training (VAT) techniques in PyTorch☆17Jul 19, 2022Updated 4 years ago
- Dynamic Telegram Trading Bot☆20Feb 21, 2025Updated last year
- Starlight: A Kernel Optimizer for GPU Processing☆17Jan 10, 2024Updated 2 years ago
- Manifold-Mixup implementation for fastai V1☆19Oct 1, 2020Updated 6 years ago
- A serverless plugin that replaces 'latest' version tag to actual lambda layer version☆25Oct 6, 2026Updated last week
- Federated learning using a mixture of experts☆17Feb 16, 2021Updated 5 years ago
- Minimal implementation of the Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models paper (ArXiv 20232401.01335)☆29Mar 1, 2024Updated 2 years ago
- A collection of optimizers, some arcane others well known, for Flax.☆29Aug 6, 2021Updated 5 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- A collection of different ways to implement accessing and modifying internal model activations for LLMs☆24Oct 18, 2024Updated last year
- Official repository for Parallax (Parameterized Local Linear Attention)☆70Jul 30, 2026Updated 2 months ago
- Pytorch implementation of HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models☆28Mar 22, 2024Updated 2 years ago
- Exploring finetuning public checkpoints on filter 8K sequences on Pile☆116Mar 22, 2023Updated 3 years ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs with LoRA support.☆30Feb 19, 2024Updated 2 years ago
- Code for "Accelerating Training with Neuron Interaction and Nowcasting Networks" [ICLR 2025]☆31Feb 20, 2026Updated 7 months ago
- Towards Understanding the Mixture-of-Experts Layer in Deep Learning☆37Dec 12, 2023Updated 2 years ago