fw-ai / benchmarkLinks
Benchmark suite for LLMs from Fireworks.ai
☆75Updated 2 weeks ago
Alternatives and similar repositories for benchmark
Users that are interested in benchmark are comparing it to the libraries listed below
Sorting:
- ☆52Updated 6 months ago
- ☆53Updated 8 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 5 months ago
- LLM Serving Performance Evaluation Harness☆78Updated 3 months ago
- ☆99Updated this week
- vLLM performance dashboard☆30Updated last year
- ☆118Updated last year
- ☆193Updated 3 weeks ago
- KV cache compression for high-throughput LLM inference☆129Updated 3 months ago
- Inference server benchmarking tool☆67Updated last month
- Boosting 4-bit inference kernels with 2:4 Sparsity☆75Updated 8 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆98Updated this week
- Easy and Efficient Quantization for Transformers☆198Updated 3 months ago
- The driver for LMCache core to run in vLLM☆41Updated 3 months ago
- Data preparation code for Amber 7B LLM☆90Updated last year
- Experiments on speculative sampling with Llama models☆126Updated last year
- ☆119Updated last year
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆11Updated last year
- ☆46Updated 11 months ago
- A safetensors extension to efficiently store sparse quantized tensors on disk☆117Updated this week
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆121Updated 11 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆263Updated 7 months ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆95Updated last year
- vLLM Router☆28Updated last year
- Example of applying CUDA graphs to LLaMA-v2☆12Updated last year
- Code for data-aware compression of DeepSeek models☆31Updated last month
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆59Updated 7 months ago
- Cascade Speculative Drafting☆29Updated last year
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- ☆85Updated 2 months ago