opendatahub-io / vllm-tgis-adapterLinks
vLLM adapter for a TGIS-compatible gRPC server.
β37Updated this week
Alternatives and similar repositories for vllm-tgis-adapter
Users that are interested in vllm-tgis-adapter are comparing it to the libraries listed below
Sorting:
- Train, tune, and infer Bamba modelβ131Updated 2 months ago
- π· Build compute kernelsβ106Updated last week
- DPO, but faster πβ44Updated 8 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language modelsβ83Updated 3 months ago
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainingsβ76Updated last month
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS β¦β61Updated 10 months ago
- Code for KaLM-Embedding modelsβ91Updated last month
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTOβ¦β56Updated 2 weeks ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundryβ42Updated last year
- IBM development fork of https://github.com/huggingface/text-generation-inferenceβ61Updated 3 months ago
- Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.β145Updated this week
- Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.β103Updated last year
- Repository for CPU Kernel Generation for LLM Inferenceβ26Updated 2 years ago
- Benchmark suite for LLMs from Fireworks.aiβ80Updated 3 weeks ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.β83Updated this week
- β48Updated 11 months ago
- Data preparation code for CrystalCoder 7B LLMβ45Updated last year
- Memory optimized Mixture of Expertsβ54Updated last month
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unslothβ167Updated this week
- Simple high-throughput inference libraryβ127Updated 3 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMsβ266Updated 10 months ago
- Make triton easierβ47Updated last year
- vLLM Routerβ39Updated last year
- This is a fork of SGLang for hip-attention integration. Please refer to hip-attention for detail.β16Updated last week
- β33Updated 2 months ago
- β38Updated last year
- β41Updated 3 months ago
- A repository for research on medium sized language models.β78Updated last year
- Official implementation for Training LLMs with MXFP4β77Updated 4 months ago
- Load compute kernels from the Hubβ244Updated last week