backprop-ai / vllm-benchmarkLinks
Benchmarking the serving capabilities of vLLM
☆46Updated 10 months ago
Alternatives and similar repositories for vllm-benchmark
Users that are interested in vllm-benchmark are comparing it to the libraries listed below
Sorting:
- ☆54Updated 7 months ago
- Self-host LLMs with vLLM and BentoML☆123Updated this week
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆137Updated 11 months ago
- ☆53Updated last year
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆67Updated last year
- Simple examples using Argilla tools to build AI☆53Updated 7 months ago
- EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language M…☆225Updated 7 months ago
- SGLang is fast serving framework for large language models and vision language models.☆23Updated 4 months ago
- Benchmark suite for LLMs from Fireworks.ai☆76Updated 2 weeks ago
- ☆267Updated 2 weeks ago
- LLM inference in C/C++☆77Updated this week
- Data preparation code for Amber 7B LLM☆91Updated last year
- Ready-to-go containerized RAG service. Implemented with text-embedding-inference + Qdrant/LanceDB.☆65Updated 6 months ago
- ☆62Updated 2 months ago
- Comparison of Language Model Inference Engines☆217Updated 6 months ago
- Open Source Text Embedding Models with OpenAI Compatible API☆153Updated 11 months ago
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆106Updated 2 months ago
- Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.☆42Updated 11 months ago
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆77Updated 8 months ago
- GPT-4 Level Conversational QA Trained In a Few Hours☆62Updated 10 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆87Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆129Updated last month
- ☆159Updated this week
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆71Updated 7 months ago
- Inference server benchmarking tool☆74Updated 2 months ago
- experiments with inference on llama☆104Updated last year
- 1.58-bit LLaMa model☆81Updated last year
- ☆66Updated last year
- 🔧 Compare how Agent systems perform on several benchmarks. 📊🚀☆98Updated 8 months ago
- 🔎 A deep-dive into HyDE for Advanced LLM RAG + 💡 Introducing AutoHyDE, a semi-supervised framework to improve the effectiveness, covera…☆32Updated last year