opendatahub-io / vllm-tgis-adapterLinks
vLLM adapter for a TGIS-compatible gRPC server.
☆47Updated this week
Alternatives and similar repositories for vllm-tgis-adapter
Users that are interested in vllm-tgis-adapter are comparing it to the libraries listed below
Sorting:
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆190Updated last week
- 👷 Build compute kernels☆213Updated this week
- Train, tune, and infer Bamba model☆138Updated 7 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆106Updated 8 months ago
- Google TPU optimizations for transformers models☆132Updated last month
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆245Updated this week
- Benchmark suite for LLMs from Fireworks.ai☆84Updated last month
- LM engine is a library for pretraining/finetuning LLMs☆110Updated last week
- Memory optimized Mixture of Experts☆72Updated 5 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆62Updated 4 months ago
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainings☆114Updated 5 months ago
- DPO, but faster 🚀☆46Updated last year
- Simple high-throughput inference library☆155Updated 8 months ago
- Simple & Scalable Pretraining for Neural Architecture Research☆306Updated last month
- Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.☆147Updated 2 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆269Updated this week
- Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research☆277Updated this week
- Tooling for exact and MinHash deduplication of large-scale text datasets☆52Updated this week
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆136Updated last year
- Official implementation for Training LLMs with MXFP4☆116Updated 8 months ago
- Fused Qwen3 MoE layer for faster training, compatible with Transformers, LoRA, bnb 4-bit quant, Unsloth. Also possible to train LoRA over…☆225Updated this week
- ☆63Updated 8 months ago
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆37Updated 3 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆59Updated 3 months ago
- NanoGPT (124M) quality in 2.67B tokens☆28Updated 4 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- Load compute kernels from the Hub☆376Updated this week
- A collection of reproducible inference engine benchmarks☆38Updated 8 months ago
- Utils for Unsloth https://github.com/unslothai/unsloth☆187Updated this week
- ☆82Updated last month