inclusionAI / asystem-awexLinks
A high-performance RL training-inference weight synchronization framework, designed to enable second-level parameter updates from training to inference in RL workflows
☆109Updated last week
Alternatives and similar repositories for asystem-awex
Users that are interested in asystem-awex are comparing it to the libraries listed below
Sorting:
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆256Updated last week
- Toolchain built around the Megatron-LM for Distributed Training☆79Updated last week
- An early research stage expert-parallel load balancer for MoE models based on linear programming.☆456Updated 3 weeks ago
- KV cache store for distributed LLM inference☆372Updated last month
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆247Updated this week
- LLM Serving Performance Evaluation Harness☆82Updated 9 months ago
- The driver for LMCache core to run in vLLM☆59Updated 10 months ago
- torchcomms: a modern PyTorch communications API☆302Updated this week
- ☆564Updated this week
- Efficient and easy multi-instance LLM serving☆517Updated 3 months ago
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆66Updated last year
- JAX backend for SGL☆191Updated this week
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆203Updated last year
- [NeurIPS 2025] Scaling Speculative Decoding with Lookahead Reasoning☆56Updated last month
- ☆69Updated 3 months ago
- Checkpoint-engine is a simple middleware to update model weights in LLM inference engines☆858Updated this week
- ☆329Updated last month
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆539Updated this week
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆135Updated last year
- ☆97Updated 8 months ago
- Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond☆701Updated 2 weeks ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inference☆345Updated this week
- HuggingFace conversion and training library for Megatron-based models☆270Updated this week
- Materials for learning SGLang☆682Updated 2 weeks ago
- Allow torch tensor memory to be released and resumed later☆187Updated 2 weeks ago
- Perplexity open source garden for inference technology☆295Updated last week
- A minimal implementation of vllm.☆62Updated last year
- Perplexity GPU Kernels☆536Updated last month
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆73Updated this week
- KV cache compression for high-throughput LLM inference☆146Updated 10 months ago