PrimeIntellect-ai / diloco_simpleLinks
torch implementation of diloco
☆22Updated last year
Alternatives and similar repositories for diloco_simple
Users that are interested in diloco_simple are comparing it to the libraries listed below
Sorting:
- ☆47Updated 2 years ago
- train with kittens!☆63Updated last year
- Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"☆149Updated 2 years ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆148Updated last year
- ☆83Updated 2 years ago
- nanoGPT-like codebase for LLM training☆113Updated 3 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆131Updated last year
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆252Updated last year
- ☆54Updated last year
- ☆52Updated last month
- ☆27Updated 2 years ago
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainings☆114Updated 6 months ago
- Triton-based implementation of Sparse Mixture of Experts.☆263Updated 4 months ago
- ☆53Updated last year
- ☆15Updated 6 months ago
- Simple high-throughput inference library☆155Updated 8 months ago
- This repository contains code for the MicroAdam paper.☆22Updated last year
- A MAD laboratory to improve AI architecture designs 🧪☆137Updated last year
- ☆124Updated last year
- Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"☆247Updated 8 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Updated last year
- A place to store reusable transformer components of my own creation or found on the interwebs☆72Updated 3 weeks ago
- Official implementation for Training LLMs with MXFP4☆118Updated 9 months ago
- Muon fsdp 2☆52Updated 5 months ago
- Inference code for LLaMA models in JAX☆120Updated last year
- One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation☆46Updated 3 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆93Updated last year
- ☆219Updated last year
- Using FlexAttention to compute attention with different masking patterns☆47Updated last year