Alternatives and similar repositories for vllm-env

Users that are interested in vllm-env are comparing it to the libraries listed below

• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆38Updated 7 months ago
• hpcaitech / CachedEmbedding
  A memory efficient DLRM training solution using ColossalAI
  ☆106Updated 3 years ago
• ninehills / langeval
  Evaluation for AI apps and agent
  ☆43Updated last year
• tensorchord / modelz-ChatGLM
  Deploy ChatGLM on Modelz
  ☆16Updated 2 years ago
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆278Updated 2 years ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆78Updated last year
• qdrant / bm42_eval
  Evaluation of bm42 sparse indexing algorithm
  ☆72Updated last year
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated 5 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆84Updated last week
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated 2 years ago
• skypilot-org / sky-llama
  ☆28Updated 2 years ago
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆85Updated 2 years ago
• OpenBuddy / GrandSage
  ☆16Updated last year
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆130Updated 2 months ago
• MooreThreads / TurboRAG
  ☆90Updated last year
• stanford-oval / ovalchat
  OVALChat is a customizable Web app aimed at conducting user studies with chatbots
  ☆28Updated last year
• myscale / vector-db-benchmark
  Framework for benchmarking fully-managed vector databases
  ☆80Updated last year
• CodeGeeX / codegeex-fastertransformer
  fastertransformer for codegeex model
  ☆65Updated 2 years ago
• bentoml / llm-bench
  ☆56Updated last year
• microsoft / batch-inference
  Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
  ☆106Updated last year
• fixie-ai / ai-benchmarks
  Benchmarking suite for popular AI APIs
  ☆88Updated 9 months ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆93Updated last year
• cxcscmu / RAGViz
  Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]
  ☆88Updated 10 months ago
• hpcaitech / SkyComputing
  Sky Computing: Accelerating Geo-distributed Computing in Federated Learning
  ☆91Updated 3 years ago
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆107Updated last year
• hellangleZ / Qwen3_autothink_adapter
  Implemented a script that automatically adjusts Qwen3's inference and non-inference capabilities, based on an OpenAI-like API. The infere…
  ☆22Updated 6 months ago
• THUDM / Multilingual-GLM
  The multilingual variant of GLM, a general language model trained with autoregressive blank infilling objective
  ☆62Updated 3 years ago
• zhuzilin / faster-nougat
  Implementation of nougat that focuses on processing pdf locally.
  ☆83Updated 10 months ago
• karminski / streaming-json-py
  A streamlined, user-friendly JSON streaming preprocessor, crafted in Python.
  ☆110Updated last year
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆103Updated last year