Alternatives and similar repositories for vllm-env

Users that are interested in vllm-env are comparing it to the libraries listed below

• hpcaitech / CachedEmbedding
  A memory efficient DLRM training solution using ColossalAI
  ☆105Updated 3 years ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆38Updated 9 months ago
• OpenBuddy / GrandSage
  ☆16Updated last year
• tensorchord / modelz-ChatGLM
  Deploy ChatGLM on Modelz
  ☆16Updated 2 years ago
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated 7 months ago
• ninehills / langeval
  Evaluation for AI apps and agent
  ☆44Updated 2 years ago
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆279Updated 2 years ago
• CodeGeeX / codegeex-fastertransformer
  fastertransformer for codegeex model
  ☆65Updated 2 years ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated 4 months ago
• skypilot-org / sky-llama
  ☆28Updated 2 years ago
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆84Updated 2 years ago
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated 2 years ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆86Updated 2 weeks ago
• zhisbug / ray-scalable-ml-design
  Some microbenchmarks and design docs before commencement
  ☆12Updated 4 years ago
• vtuber-plan / langport
  Langport is a language model inference service
  ☆94Updated last year
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆78Updated last year
• bentoml / llm-bench
  ☆56Updated last year
• qdrant / bm42_eval
  Evaluation of bm42 sparse indexing algorithm
  ☆72Updated last year
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆30Updated 10 months ago
• OrchardUniverse / litchi
  Yet another coding assistant powered by LLM.
  ☆16Updated last year
• kailums / flash-attention-rocm
  Fast and memory-efficient exact attention ported to rocm
  ☆13Updated 2 years ago
• zhuzilin / faster-nougat
  Implementation of nougat that focuses on processing pdf locally.
  ☆84Updated last year
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆103Updated last year
• fixie-ai / ai-benchmarks
  Benchmarking suite for popular AI APIs
  ☆88Updated 11 months ago
• leloykun / llama2.cpp
  Inference Llama 2 in one file of pure C++
  ☆87Updated 2 years ago
• hellangleZ / Qwen3_autothink_adapter
  Implemented a script that automatically adjusts Qwen3's inference and non-inference capabilities, based on an OpenAI-like API. The infere…
  ☆22Updated 8 months ago
• shrimantasatpati / Microsoft-Phi-2-Streamlit
  Microsoft Phi 2 Streamlit App, deployed on HuggingFace Spaces is based on the Microsoft Phi 2 small language model (SLM) for text generat…
  ☆14Updated last year
• leoguillaume / vllm-embedding
  Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.
  ☆44Updated last year
• tensorchord / modelz-llm
  OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)
  ☆275Updated 2 years ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆90Updated 4 months ago