A codebase for pretraining multi-billion-scale sparse GPTs.
☆30Feb 9, 2026Updated 9 months ago
Alternatives and similar repositories for Sparse-GPT-Pretraining
Users that are interested in Sparse-GPT-Pretraining are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆13Updated this week
- Causal Attention with Lookahead Keys☆28Sep 26, 2025Updated last year
- Experiments for the spectral neuron paper☆23Sep 5, 2026Updated 2 months ago
- CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs☆258Nov 14, 2026Updated last week
- Official repository for "SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space"☆33May 7, 2026Updated 6 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- GPU-accelerated LLM Training Simulator☆23Jun 26, 2025Updated last year
- A survey of modern quantization formats (e.g., MXFP8, NVFP4) and inference optimization tools (e.g., TorchAO, GemLite), illustrated throu…☆12Nov 11, 2025Updated last year
- ☆29Jan 19, 2026Updated 10 months ago
- Two case studies: megakernels roughly on par with CUDA Graphs + PDL. Code and baselines for further research.☆13Nov 6, 2026Updated 2 weeks ago
- ☆37Aug 7, 2025Updated last year
- Open source reference implementation for a simple, metamodel-based Hyperliquid portfolio rebalancer☆12Dec 10, 2025Updated 11 months ago
- Disaggregated Prefill-Decode LLM serving engine on mini-sglang with a proved KV page state machine (PdLedger)☆25Nov 11, 2026Updated last week
- An experimental communicating attention kernel based on DeepEP.☆34Jul 29, 2025Updated last year
- Accelerating MoE with IO and Tile-aware Optimizations☆791Aug 29, 2026Updated 2 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆11Nov 20, 2024Updated 2 years ago
- ☆25Jun 12, 2025Updated last year
- AES Tutorial 2021, Deep Learning for Audio☆14Aug 28, 2024Updated 2 years ago
- Simple (and currently incomplete) Python wrapper around the Opentrons HTTP API☆11Nov 26, 2025Updated 11 months ago
- ☆12Jul 8, 2026Updated 4 months ago
- Learn about PTX instructions ldmatrix and stmatrix☆12Sep 5, 2025Updated last year
- Distil SN97 — Competitive Model Distillation on Bittensor☆35May 20, 2026Updated 6 months ago
- ☆25May 23, 2026Updated 5 months ago
- Source code for our paper: "Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents".☆26Feb 20, 2026Updated 9 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A port of OneLoneCoder's PixelGameEngine in C#☆14Jun 20, 2026Updated 5 months ago
- A Quirky Assortment of CuTe Kernels☆1,196Oct 30, 2026Updated 3 weeks ago
- Official Repository for "The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm" (ICLR 2026)☆13Apr 16, 2026Updated 7 months ago
- Research work aimed at addressing the problem of modeling infinite-length context☆53Dec 18, 2025Updated 11 months ago
- ☆14Nov 29, 2024Updated last year
- Code for the papers: “Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling” and “Adaptive Block-Scaled Data Types”☆213Apr 21, 2026Updated 7 months ago
- wolfbench.ai · Wolfram Ravenwolf's Five-Metric Framework · based on Terminal-Bench 2.0☆12Apr 29, 2026Updated 6 months ago
- Shared memory IPC on BMC☆15Oct 8, 2026Updated last month
- Convert any folder into a pdf. Makes it easy to print with a printer. My first vibe coded application with Claude Code.☆16Mar 14, 2026Updated 8 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 🔥 A minimal training framework for scaling FLA models☆429Apr 22, 2026Updated 6 months ago
- Reverse engineering notes. Personal reference only. Everything here is a best-guess reconstruction.☆138Aug 11, 2026Updated 3 months ago
- Weak-to-Strong Consistency Learning for Semi-Supervised Cloud Detection☆12Oct 15, 2025Updated last year
- ☆15Dec 26, 2025Updated 10 months ago
- Very fast loader for libsvm / svmlight sparse data files in Python (Cython): SciPy sparse matrices, gzip and bz2 input, an alternative to…☆10Oct 20, 2026Updated last month
- ☆17May 18, 2026Updated 6 months ago
- Official PyTorch implementation of "Latent Reasoning in TRMs is Secretly a Policy Improvement Operator" (ICML 2026)☆27May 29, 2026Updated 5 months ago
Are these results useful?