kemingy / vllm-envLinks
setup the env for vllm users
☆16Updated last year
Alternatives and similar repositories for vllm-env
Users that are interested in vllm-env are comparing it to the libraries listed below
Sorting:
- Evaluation of bm42 sparse indexing algorithm☆68Updated last year
- Benchmark suite for LLMs from Fireworks.ai☆83Updated this week
- A memory efficient DLRM training solution using ColossalAI☆106Updated 2 years ago
- A collection of reproducible inference engine benchmarks☆33Updated 5 months ago
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated 2 years ago
- Sentence Embedding as a Service☆15Updated 3 months ago
- fastertransformer for codegeex model☆65Updated 2 years ago
- Deploy ChatGLM on Modelz☆16Updated 2 years ago
- ☆85Updated 2 years ago
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆275Updated last year
- Some microbenchmarks and design docs before commencement☆12Updated 4 years ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆72Updated last year
- Evaluation for AI apps and agent☆43Updated last year
- ☆16Updated last year
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.☆29Updated 6 months ago
- ☆51Updated last year
- Open Implementations of LLM Analyses☆107Updated 11 months ago
- LLMs as Collaboratively Edited Knowledge Bases☆45Updated last year
- A lightweight script for processing HTML page to markdown format with support for code blocks☆80Updated last year
- Tutorial for LLM developers about engine design, service deployment, evaluation/benchmark, etc. Provide a C/S style optimized LLM inferen…☆19Updated 2 years ago
- Data preparation code for Amber 7B LLM☆93Updated last year
- ☆31Updated last year
- The multilingual variant of GLM, a general language model trained with autoregressive blank infilling objective☆62Updated 2 years ago
- Evaluating tool-augmented LLMs in conversation settings☆88Updated last year
- ☆64Updated 6 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated last week
- vLLM Router☆45Updated last year
- Implemented a script that automatically adjusts Qwen3's inference and non-inference capabilities, based on an OpenAI-like API. The infere…☆22Updated 4 months ago
- ☆28Updated 2 years ago
- Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]☆85Updated 8 months ago