opendatahub-io / vllm-tgis-adapterLinks
vLLM adapter for a TGIS-compatible gRPC server.
β41Updated this week
Alternatives and similar repositories for vllm-tgis-adapter
Users that are interested in vllm-tgis-adapter are comparing it to the libraries listed below
Sorting:
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainingsβ88Updated 3 months ago
- π· Build compute kernelsβ163Updated this week
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS β¦β60Updated last year
- Train, tune, and infer Bamba modelβ135Updated 4 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language modelsβ95Updated 5 months ago
- DPO, but faster πβ45Updated 10 months ago
- Simple high-throughput inference libraryβ149Updated 5 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundryβ42Updated last year
- Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.β147Updated 2 weeks ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decodingβ130Updated 10 months ago
- Benchmark suite for LLMs from Fireworks.aiβ83Updated 2 weeks ago
- Make triton easierβ48Updated last year
- Self-host LLMs with LMDeploy and BentoMLβ21Updated 3 months ago
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unslothβ197Updated last week
- IBM development fork of https://github.com/huggingface/text-generation-inferenceβ61Updated last month
- β58Updated 5 months ago
- Nexusflow function call, tool use, and agent benchmarks.β29Updated 10 months ago
- KV cache compression for high-throughput LLM inferenceβ143Updated 8 months ago
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.β36Updated 3 weeks ago
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLMβ60Updated last week
- Memory optimized Mixture of Expertsβ68Updated 3 months ago
- Repository for CPU Kernel Generation for LLM Inferenceβ26Updated 2 years ago
- β65Updated 7 months ago
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTOβ¦β56Updated last week
- β52Updated last year
- Accelerating your LLM training to full speed! Made with β€οΈ by ServiceNow Researchβ254Updated last week
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inferenceβ68Updated 10 months ago
- A repository for research on medium sized language models.β78Updated last year
- β78Updated 2 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optunaβ58Updated last week