huggingface / picotron_tutorialLinks
β206Updated 5 months ago
Alternatives and similar repositories for picotron_tutorial
Users that are interested in picotron_tutorial are comparing it to the libraries listed below
Sorting:
- An extension of the nanoGPT repository for training small MOE models.β163Updated 4 months ago
- π Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flashβ¦β258Updated last week
- Best practices & guides on how to write distributed pytorch training codeβ460Updated 5 months ago
- Scalable toolkit for efficient model reinforcementβ558Updated this week
- Load compute kernels from the Hubβ220Updated this week
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"β506Updated 3 weeks ago
- PyTorch building blocks for the OLMo ecosystemβ269Updated this week
- A repository to unravel the language of GPUs, making their kernel conversations easy to understandβ188Updated 2 months ago
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, sparsβ¦β342Updated 7 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.β149Updated last month
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024β323Updated 3 months ago
- β347Updated this week
- A project to improve skills of large language modelsβ501Updated this week
- ring-attention experimentsβ146Updated 9 months ago
- Decentralized RL Training at Scaleβ400Updated this week
- Normalized Transformer (nGPT)β185Updated 8 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)β190Updated this week
- πΎ OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.β418Updated last week
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top ofβ¦β138Updated 11 months ago
- A scalable asynchronous reinforcement learning implementation with in-flight weight updates.β134Updated this week
- β162Updated last year
- Prune transformer layersβ69Updated last year
- PyTorch Single Controllerβ341Updated this week
- Understand and test language model architectures on synthetic tasks.β221Updated 3 weeks ago
- β182Updated 7 months ago
- Explorations into some recent techniques surrounding speculative decodingβ275Updated 7 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)β372Updated this week
- Tina: Tiny Reasoning Models via LoRAβ272Updated 2 months ago
- Code for studying the super weight in LLMβ114Updated 8 months ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Dayβ256Updated last year