opendatahub-io / vllm-tgis-adapterLinks
vLLM adapter for a TGIS-compatible gRPC server.
β41Updated this week
Alternatives and similar repositories for vllm-tgis-adapter
Users that are interested in vllm-tgis-adapter are comparing it to the libraries listed below
Sorting:
- π· Build compute kernelsβ155Updated this week
- Lightweight toolkit package to train and fine-tune 1.58bit Language modelsβ89Updated 4 months ago
- Benchmark suite for LLMs from Fireworks.aiβ83Updated last week
- Train, tune, and infer Bamba modelβ133Updated 4 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS β¦β60Updated 11 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decodingβ128Updated 10 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inferenceβ61Updated 2 weeks ago
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainingsβ83Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMsβ266Updated 11 months ago
- Simple high-throughput inference libraryβ139Updated 4 months ago
- Google TPU optimizations for transformers modelsβ120Updated 8 months ago
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unslothβ183Updated 2 weeks ago
- Load compute kernels from the Hubβ293Updated this week
- β43Updated 3 weeks ago
- Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.β148Updated this week
- ArcticInference: vLLM plugin for high-throughput, low-latency inferenceβ270Updated this week
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLMβ50Updated last week
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTOβ¦β56Updated 2 weeks ago
- XTR: Rethinking the Role of Token Retrieval in Multi-Vector Retrievalβ58Updated last year
- Code for KaLM-Embedding modelsβ93Updated 3 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundryβ42Updated last year
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)β219Updated this week
- Repository for CPU Kernel Generation for LLM Inferenceβ26Updated 2 years ago
- SGLang is fast serving framework for large language models and vision language models.β27Updated last month
- Data preparation code for CrystalCoder 7B LLMβ45Updated last year
- DPO, but faster πβ44Updated 10 months ago
- KV cache compression for high-throughput LLM inferenceβ141Updated 8 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked promptsβ24Updated last year
- Make triton easierβ47Updated last year
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.β36Updated 2 months ago