opendatahub-io / vllm-tgis-adapterLinks
vLLM adapter for a TGIS-compatible gRPC server.
β34Updated this week
Alternatives and similar repositories for vllm-tgis-adapter
Users that are interested in vllm-tgis-adapter are comparing it to the libraries listed below
Sorting:
- π· Build compute kernelsβ87Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inferenceβ61Updated 2 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language modelsβ82Updated 2 months ago
- Benchmark suite for LLMs from Fireworks.aiβ76Updated this week
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainingsβ68Updated last week
- Google TPU optimizations for transformers modelsβ117Updated 6 months ago
- Train, tune, and infer Bamba modelβ130Updated 2 months ago
- Code for KaLM-Embedding modelsβ86Updated last month
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)β195Updated this week
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTOβ¦β56Updated last week
- Load compute kernels from the Hubβ220Updated this week
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS β¦β61Updated 9 months ago
- β48Updated 11 months ago
- Simple high-throughput inference libraryβ125Updated 2 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decodingβ123Updated 8 months ago
- DPO, but faster πβ44Updated 8 months ago
- Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.β141Updated this week
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.β83Updated 2 weeks ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optunaβ55Updated 6 months ago
- Memory optimized Mixture of Expertsβ51Updated last week
- β37Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentβ60Updated 11 months ago
- Data preparation code for CrystalCoder 7B LLMβ45Updated last year
- Data preparation code for Amber 7B LLMβ91Updated last year
- Crispy reranking models by Mixedbreadβ33Updated 3 weeks ago
- Accelerating your LLM training to full speed! Made with β€οΈ by ServiceNow Researchβ218Updated this week
- vLLM Routerβ39Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMsβ266Updated 9 months ago
- XTR: Rethinking the Role of Token Retrieval in Multi-Vector Retrievalβ52Updated last year
- Repository containing the SPIN experiments on the DIBT 10k ranked promptsβ24Updated last year