changjonathanc / flex-nano-vllmLinks
FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.
☆300Updated 2 months ago
Alternatives and similar repositories for flex-nano-vllm
Users that are interested in flex-nano-vllm are comparing it to the libraries listed below
Sorting:
- ☆224Updated last week
- Load compute kernels from the Hub☆308Updated this week
- An extension of the nanoGPT repository for training small MOE models.☆205Updated 7 months ago
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆194Updated 4 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆344Updated 5 months ago
- Dion optimizer algorithm☆374Updated last month
- Simple & Scalable Pretraining for Neural Architecture Research☆297Updated 2 months ago
- 👷 Build compute kernels☆163Updated this week
- ☆868Updated 2 weeks ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆270Updated 3 months ago
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆347Updated 10 months ago
- Exploring Applications of GRPO☆248Updated 2 months ago
- Async RL Training at Scale☆722Updated this week
- ☆174Updated last year
- PyTorch-native post-training at scale☆388Updated this week
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆197Updated 4 months ago
- PyTorch building blocks for the OLMo ecosystem☆311Updated this week
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆248Updated 9 months ago
- Normalized Transformer (nGPT)☆192Updated 11 months ago
- Efficient LLM Inference over Long Sequences☆390Updated 4 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆436Updated last week
- Pytorch DTensor native training library for LLMs/VLMs with OOTB Hugging Face support☆141Updated this week
- PTX-Tutorial Written Purely By AIs (Deep Research of Openai and Claude 3.7)☆66Updated 7 months ago
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"☆544Updated 3 weeks ago
- A scalable asynchronous reinforcement learning implementation with in-flight weight updates.☆260Updated last week
- ☆446Updated 2 months ago
- The evaluation framework for training-free sparse attention in LLMs☆102Updated 2 weeks ago
- rl from zero pretrain, can it be done? yes.☆279Updated last month
- Memory optimized Mixture of Experts☆68Updated 3 months ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆168Updated 4 months ago