runpod-workers / worker-sglangLinks
SGLang is fast serving framework for large language models and vision language models.
☆29Updated last week
Alternatives and similar repositories for worker-sglang
Users that are interested in worker-sglang are comparing it to the libraries listed below
Sorting:
- Google TPU optimizations for transformers models☆120Updated 9 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆266Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆202Updated last year
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆139Updated last year
- vLLM adapter for a TGIS-compatible gRPC server.☆41Updated this week
- ☆51Updated last year
- ☆136Updated 2 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆58Updated this week
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆92Updated 5 months ago
- Simple high-throughput inference library☆147Updated 5 months ago
- Utils for Unsloth https://github.com/unslothai/unsloth☆160Updated this week
- Benchmark suite for LLMs from Fireworks.ai☆82Updated last week
- experiments with inference on llama☆103Updated last year
- ☆136Updated last year
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainings