huggingface / picotron_tutorialLinks
β193Updated 4 months ago
Alternatives and similar repositories for picotron_tutorial
Users that are interested in picotron_tutorial are comparing it to the libraries listed below
Sorting:
- π Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flashβ¦β253Updated this week
- An extension of the nanoGPT repository for training small MOE models.β152Updated 3 months ago
- Scalable toolkit for efficient model reinforcementβ438Updated this week
- Load compute kernels from the Hubβ191Updated this week
- The simplest, fastest repository for training/finetuning medium-sized GPTs.β134Updated this week
- A repository to unravel the language of GPUs, making their kernel conversations easy to understandβ185Updated 3 weeks ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"β239Updated 4 months ago
- Triton-based implementation of Sparse Mixture of Experts.β219Updated 6 months ago
- β78Updated 11 months ago
- A scalable asynchronous reinforcement learning implementation with in-flight weight updates.β125Updated this week
- ring-attention experimentsβ144Updated 8 months ago
- Understand and test language model architectures on synthetic tasks.β217Updated 2 weeks ago
- πΎ OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.β379Updated 2 weeks ago
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, sparsβ¦β339Updated 6 months ago
- Normalized Transformer (nGPT)β184Updated 7 months ago
- LLM KV cache compression made easyβ520Updated this week
- Efficient LLM Inference over Long Sequencesβ378Updated 2 weeks ago
- β159Updated last year
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top ofβ¦β137Updated 10 months ago
- PyTorch Single Controllerβ218Updated this week
- prime-rl is a codebase for decentralized async RL training at scaleβ341Updated this week
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024β311Updated last month
- A MAD laboratory to improve AI architecture designs π§ͺβ120Updated 6 months ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.β556Updated last week
- PyTorch building blocks for the OLMo ecosystemβ238Updated this week
- Best practices & guides on how to write distributed pytorch training codeβ441Updated 4 months ago
- Scalable and Performant Data Loadingβ277Updated this week
- This repository contains the experimental PyTorch native float8 training UXβ224Updated 10 months ago
- Cataloging released Triton kernels.β238Updated 5 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)β119Updated this week