Alternatives and similar repositories for vllm-env

Users that are interested in vllm-env are comparing it to the libraries listed below

• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆32Updated 3 months ago
• CodeGeeX / codegeex-fastertransformer
  fastertransformer for codegeex model
  ☆64Updated 2 years ago
• hpcaitech / CachedEmbedding
  A memory efficient DLRM training solution using ColossalAI
  ☆105Updated 2 years ago
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated last month
• laramohan / wikillm
  LLMs as Collaboratively Edited Knowledge Bases
  ☆45Updated last year
• qdrant / bm42_eval
  Evaluation of bm42 sparse indexing algorithm
  ☆68Updated last year
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated 2 years ago
• ninehills / langeval
  Evaluation for AI apps and agent
  ☆42Updated last year
• LMCache / demo
  ☆20Updated 3 months ago
• OpenBuddy / GrandSage
  ☆16Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆76Updated this week
• LLM-inference-router / vllm-router
  vLLM Router
  ☆39Updated last year
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆84Updated last year
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆69Updated last year
• zhisbug / ray-scalable-ml-design
  Some microbenchmarks and design docs before commencement
  ☆12Updated 4 years ago
• microsoft / ToolTalk
  Evaluating tool-augmented LLMs in conversation settings
  ☆85Updated last year
• allenai / recoma
  Reasoning by Communicating with Agents
  ☆29Updated 3 months ago
• vtuber-plan / langport
  Langport is a language model inference service
  ☆93Updated 10 months ago
• nlpodyssey / rwkv
  RWKV (Receptance Weighted Key Value) is a RNN with Transformer-level performance
  ☆41Updated 2 years ago
• tensorchord / modelz-ChatGLM
  Deploy ChatGLM on Modelz
  ☆15Updated 2 years ago
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆105Updated 9 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆40Updated last year
• OrchardUniverse / litchi
  Yet another coding assistant powered by LLM.
  ☆16Updated 10 months ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆87Updated last month
• mlc-ai / llm-perf-bench
  ☆120Updated last year
• skypilot-org / sky-llama
  ☆28Updated 2 years ago
• dust-tt / llama-ssp
  Experiments on speculative sampling with Llama models
  ☆128Updated 2 years ago
• VikParuchuri / classified
  Score LLM pretraining data with classifiers
  ☆55Updated last year
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆128Updated 3 weeks ago
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆29Updated 4 months ago