Alternatives and similar repositories for vllm-env

Users that are interested in vllm-env are comparing it to the libraries listed below

• hpcaitech / CachedEmbedding
  A memory efficient DLRM training solution using ColossalAI
  ☆106Updated 2 years ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆32Updated 4 months ago
• tensorchord / ai-infra-statistics
  This repository contains statistics about the AI Infrastructure products.
  ☆17Updated 6 months ago
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated 2 months ago
• CodeGeeX / codegeex-fastertransformer
  fastertransformer for codegeex model
  ☆64Updated 2 years ago
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆85Updated 2 years ago
• ninehills / langeval
  Evaluation for AI apps and agent
  ☆43Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆83Updated last week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆71Updated last year
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated 2 years ago
• nyunAI / PruneGPT
  ☆51Updated last year
• hpcaitech / Titans
  A collection of models built with ColossalAI
  ☆32Updated 2 years ago
• SivilTaram / code-html-to-markdown
  A lightweight script for processing HTML page to markdown format with support for code blocks
  ☆80Updated last year
• qdrant / bm42_eval
  Evaluation of bm42 sparse indexing algorithm
  ☆68Updated last year
• OpenBuddy / GrandSage
  ☆16Updated last year
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆273Updated last year
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated last year
• tensorchord / modelz-ChatGLM
  Deploy ChatGLM on Modelz
  ☆16Updated 2 years ago
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆106Updated 11 months ago
• microsoft / ToolTalk
  Evaluating tool-augmented LLMs in conversation settings
  ☆87Updated last year
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• skypilot-org / sky-llama
  ☆28Updated 2 years ago
• dust-tt / llama-ssp
  Experiments on speculative sampling with Llama models
  ☆128Updated 2 years ago
• FreedomIntelligence / GPT-API-Accelerate
  The "GPT-API-Accelerate" project provides a set of Python classes for accelerating the process of generating responses to prompts using t…
  ☆23Updated 11 months ago
• sambanova / generative_data_prep
  ☆63Updated 4 months ago
• vtuber-plan / langport
  Langport is a language model inference service
  ☆94Updated last year
• myscale / Retrieval-QA-Benchmark
  Benchmark baseline for retrieval qa applications
  ☆116Updated last year
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆45Updated last year
• cxcscmu / RAGViz
  Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]
  ☆85Updated 7 months ago
• etalab-ia / albert-models
  Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.
  ☆42Updated last year