ninehills / llm-inference-benchmarkLinks
LLM Inference benchmark
☆430Updated last year
Alternatives and similar repositories for llm-inference-benchmark
Users that are interested in llm-inference-benchmark are comparing it to the libraries listed below
Sorting:
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆249Updated last year
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆268Updated 3 months ago
- 中文版 llm-numbers☆126Updated last year
- ☆512Updated 2 months ago
- RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.☆926Updated this week
- Accelerate inference without tears☆367Updated last month
- Efficient AI Inference & Serving☆478Updated last year
- A high-performance inference system for large language models, designed for production environments.☆482Updated 2 weeks ago
- ☆431Updated 2 months ago
- FlagScale is a large model toolkit based on open-sourced projects.☆407Updated last week
- A flexible and efficient training framework for large-scale alignment tasks☆437Updated 3 weeks ago
- Best practice for training LLaMA models in Megatron-LM☆659Updated last year
- ☆177Updated this week
- ☆130Updated 10 months ago
- InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…☆411Updated 3 months ago
- LLM 推理服务性能测试☆44Updated last year
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago
- Materials for learning SGLang☆644Updated last week
- Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline mod…☆578Updated last year
- 一种任务级GPU算力分时调度的高性能深度学习训练平台☆712Updated 2 years ago
- ☆22Updated 2 years ago
- ☆53Updated last year
- Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs…☆578Updated 2 months ago
- ☆967Updated 9 months ago
- C++ implementation of Qwen-LM☆606Updated 11 months ago
- GLake: optimizing GPU memory management and IO transmission.☆489Updated 7 months ago
- 中文Mixtral-8x7B（Chinese-Mixtral-8x7B）☆655Updated last year
- ☆65Updated last week
- [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLM, VLM, and video generation models.☆622Updated last week
- The official codes for "Aurora: Activating chinese chat capability for Mixtral-8x7B sparse Mixture-of-Experts through Instruction-Tuning"☆265Updated last year