huggingface / picotron_tutorialLinks
☆224Updated last week
Alternatives and similar repositories for picotron_tutorial
Users that are interested in picotron_tutorial are comparing it to the libraries listed below
Sorting:
- FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.☆313Updated last month
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆271Updated last week
- An extension of the nanoGPT repository for training small MOE models.☆215Updated 8 months ago
- Load compute kernels from the Hub☆337Updated last week
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆196Updated 6 months ago
- A scalable asynchronous reinforcement learning implementation with in-flight weight updates.☆316Updated this week
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆174Updated 5 months ago
- ring-attention experiments☆160Updated last year
- Best practices & guides on how to write distributed pytorch training code☆543Updated last month
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆347Updated 7 months ago
- ☆317Updated this week
- Normalized Transformer (nGPT)☆194Updated last year
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆360Updated 11 months ago
- 👷 Build compute kernels☆190Updated this week
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆187Updated last week
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"☆561Updated last month
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆254Updated last week
- HuggingFace conversion and training library for Megatron-based models☆228Updated this week
- Efficient LLM Inference over Long Sequences☆392Updated 5 months ago
- Code for studying the super weight in LLM☆121Updated last year
- ☆222Updated 11 months ago
- Memory optimized Mixture of Experts☆69Updated 4 months ago
- PyTorch building blocks for the OLMo ecosystem☆482Updated this week
- Simple & Scalable Pretraining for Neural Architecture Research☆302Updated last month
- ☆91Updated last year
- ☆917Updated last month
- ☆177Updated last year
- ☆546Updated last year
- Understand and test language model architectures on synthetic tasks.☆240Updated 2 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆146Updated last year