anarchy-ai / llm-speed-benchmarkLinks
Benchmarking tool for assessing LLM models' performance across different hardwares
☆17Updated last year
Alternatives and similar repositories for llm-speed-benchmark
Users that are interested in llm-speed-benchmark are comparing it to the libraries listed below
Sorting:
- vLLM with support for span semantics☆18Updated last week
- Transformer GPU VRAM estimator☆66Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆52Updated last year
- Public reports detailing responses to sets of prompts by Large Language Models.☆31Updated 9 months ago
- StarListify is a Python package that classifies GitHub stars history into organized category lists based on user-defined criteria.☆28Updated 11 months ago
- ☆22Updated last year
- Realtime News and Information Eval☆14Updated last month
- Aana SDK is a powerful framework for building AI enabled multimodal applications.☆52Updated last month
- Simple high-throughput inference library☆146Updated 5 months ago
- The backend behind the LLM-Perf Leaderboard☆10Updated last year
- Visualize expert firing frequencies across sentences in the Mixtral MoE model☆18Updated last year
- EXO Gym is an open-source Python toolkit that facilitates distributed AI research.☆78Updated last month
- ☆19Updated last year
- Routing on Random Forest (RoRF)☆213Updated last year
- Benchmarking suite for popular AI APIs☆87Updated 8 months ago
- Tutorial for building LLM router☆230Updated last year
- Benchmark suite for LLMs from Fireworks.ai☆83Updated last week
- Implementation of nougat that focuses on processing pdf locally.☆83Updated 9 months ago
- Command line tool for Deep Infra cloud ML inference service☆33Updated last year
- Replace expensive LLM calls with finetunes automatically☆64Updated last year
- Quickly and securely turn any Linux box into a build and deployment assistant☆24Updated last year
- Python client library for improving your LLM app accuracy☆97Updated 8 months ago
- ☆162Updated 2 months ago
- Analyzing and scoring reasoning traces of LLMs☆46Updated last year
- Verbosity control for AI agents☆65Updated last year
- [⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI☆48Updated 3 months ago
- A collection of all available inference solutions for the LLMs☆91Updated 7 months ago
- Senna is an advanced AI-powered search engine designed to provide users with immediate answers to their queries by leveraging natural lan…☆19Updated last year
- ☆47Updated last year
- Chrome Extension for exploring Hugging Face datasets 🔎☆48Updated last year