Snowflake-Labs / vllmLinks
☆16Updated 3 weeks ago
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below
Sorting:
- Benchmark suite for LLMs from Fireworks.ai☆84Updated 3 weeks ago
- A collection of reproducible inference engine benchmarks☆38Updated 7 months ago
- ☆47Updated last year
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆257Updated this week
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆149Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆62Updated 2 months ago
- Cray-LM unified training and inference stack.☆22Updated 10 months ago
- Google TPU optimizations for transformers models☆124Updated 10 months ago
- vLLM adapter for a TGIS-compatible gRPC server.☆45Updated this week
- ArcticInference: vLLM plugin for high-throughput, low-latency inference☆345Updated this week
- Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆202Updated this week
- ☆52Updated last year
- ☆31Updated last year
- LM engine is a library for pretraining/finetuning LLMs☆77Updated last week
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- Simple high-throughput inference library☆151Updated 7 months ago
- Intel Gaudi's Megatron DeepSpeed Large Language Models for training☆15Updated 11 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆93Updated last week
- Example ML projects that use the Determined library.☆32Updated last year
- Repository for CPU Kernel Generation for LLM Inference☆27Updated 2 years ago
- experiments with inference on llama☆103Updated last year
- Make triton easier☆49Updated last year
- Train, tune, and infer Bamba model☆137Updated 6 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆267Updated last week
- Easy and Efficient Quantization for Transformers☆203Updated 5 months ago
- ☆15Updated 5 months ago
- Efficient non-uniform quantization with GPTQ for GGUF☆56Updated 2 months ago
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆65Updated 2 years ago
- ☆112Updated 3 weeks ago