volcengine / veScaleLinks
A PyTorch Native LLM Training Framework
☆819Updated 5 months ago
Alternatives and similar repositories for veScale
Users that are interested in veScale are comparing it to the libraries listed below
Sorting:
- Disaggregated serving system for Large Language Models (LLMs).☆617Updated 2 months ago
- Distributed Compiler Based on Triton for Parallel Systems☆829Updated this week
- Zero Bubble Pipeline Parallelism☆398Updated last month
- A fast communication-overlapping library for tensor/expert parallelism on GPUs.☆978Updated 3 weeks ago
- A throughput-oriented high-performance serving framework for LLMs☆825Updated 2 weeks ago
- Ring attention implementation with flash attention☆782Updated last week
- Microsoft Automatic Mixed Precision Library☆609Updated 8 months ago
- RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.☆791Updated 2 weeks ago
- GLake: optimizing GPU memory management and IO transmission.☆467Updated 2 months ago
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆519Updated 3 weeks ago
- Materials for learning SGLang☆443Updated this week
- Efficient and easy multi-instance LLM serving☆437Updated this week
- ☆593Updated last month
- Perplexity GPU Kernels☆364Updated last week
- Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline mod…☆480Updated 9 months ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆396Updated 3 weeks ago
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Se…☆705Updated 3 months ago
- FlagScale is a large model toolkit based on open-sourced projects.☆301Updated this week
- Latency and Memory Analysis of Transformer Models for Training and Inference☆429Updated 2 months ago
- FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.☆845Updated 9 months ago
- Best practice for training LLaMA models in Megatron-LM☆656Updated last year
- Fast inference from large lauguage models via speculative decoding☆762Updated 10 months ago
- FlagGems is an operator library for large language models implemented in the Triton Language.☆573Updated this week
- Pipeline Parallelism for PyTorch☆768Updated 10 months ago
- Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels☆1,292Updated this week
- NVIDIA Inference Xfer Library (NIXL)☆413Updated this week
- FlashInfer: Kernel Library for LLM Serving☆3,211Updated this week
- 📰 Must-read papers and blogs on Speculative Decoding ⚡️☆792Updated this week
- A low-latency & high-throughput serving engine for LLMs☆379Updated 3 weeks ago
- NCCL Tests☆1,149Updated 2 weeks ago