friendliai / LLMServingPerfEvaluatorLinks
☆47Updated last year
Alternatives and similar repositories for LLMServingPerfEvaluator
Users that are interested in LLMServingPerfEvaluator are comparing it to the libraries listed below
Sorting:
- ☆103Updated 2 years ago
- FMO (Friendli Model Optimizer)☆12Updated 8 months ago
- Welcome to PeriFlow CLI ☁︎☆12Updated 2 years ago
- A performance library for machine learning applications.☆184Updated last year
- ☆25Updated 2 years ago
- FriendliAI Model Hub☆91Updated 3 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆83Updated this week
- PyTorch CoreSIG☆56Updated 8 months ago
- ☆73Updated 3 months ago
- ☆54Updated 10 months ago
- QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference☆118Updated last year
- Easy and Efficient Quantization for Transformers☆203Updated 2 months ago
- ☆90Updated last year
- A low-latency & high-throughput serving engine for LLMs☆416Updated 3 months ago
- ☆12Updated 5 months ago
- Official Github repository for the SIGCOMM '24 paper "Accelerating Model Training in Multi-cluster Environments with Consumer-grade GPUs"☆71Updated last year
- [ACM EuroSys 2023] Fast and Efficient Model Serving Using Multi-GPUs with Direct-Host-Access☆57Updated last month
- Study Group of Deep Learning Compiler☆163Updated 2 years ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆416Updated 3 months ago
- ☆296Updated last week
- MIST: High-performance IoT Stream Processing☆17Updated 6 years ago
- ☆56Updated 2 years ago
- ☆24Updated 6 years ago
- NEST Compiler☆117Updated 7 months ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inference☆239Updated this week
- The Universe of Evaluation. All about the evaluation for LLMs.☆227Updated last year
- Lightweight and Parallel Deep Learning Framework☆264Updated 2 years ago
- Modular and structured prompt caching for low-latency LLM inference☆100Updated 10 months ago
- Know Your Enemy To Save Cloud Energy: Energy-Performance Characterization of Machine Learning Serving (HPCA '23)☆13Updated 3 months ago
- OwLite is a low-code AI model compression toolkit for AI models.☆50Updated 4 months ago