anarchy-ai / llm-speed-benchmarkLinks
Benchmarking tool for assessing LLM models' performance across different hardwares
☆17Updated 2 years ago
Alternatives and similar repositories for llm-speed-benchmark
Users that are interested in llm-speed-benchmark are comparing it to the libraries listed below
Sorting:
- Transformer GPU VRAM estimator☆67Updated last year
- vLLM with support for span semantics☆21Updated last week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆53Updated 2 years ago
- The backend behind the LLM-Perf Leaderboard☆11Updated last year
- Rust crates for XetHub☆75Updated last year
- Natural Language Interfaces Powered by LLMs☆95Updated last year
- Python client library for improving your LLM app accuracy☆97Updated 10 months ago
- ☆115Updated 10 months ago
- StarListify is a Python package that classifies GitHub stars history into organized category lists based on user-defined criteria.☆31Updated last year
- Drop in replacement for OpenAI, but with Open models.☆153Updated 2 years ago
- Use context-free grammars with an LLM☆175Updated last year
- Public reports detailing responses to sets of prompts by Large Language Models.☆32Updated 11 months ago
- ☆474Updated last year
- Using Large Language Models for Repo-wide Type Prediction☆112Updated 2 years ago
- Benchmark structured generation libraries☆30Updated last year
- Aider's refactoring benchmark exercises based on popular python repos☆78Updated last year
- Realtime News and Information Eval☆15Updated last month
- ReLM is a Regular Expression engine for Language Models☆107Updated 2 years ago
- Analyzing and scoring reasoning traces of LLMs☆47Updated last year
- ☆198Updated last year
- Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)☆81Updated 10 months ago
- Benchmarking suite for popular AI APIs☆89Updated 10 months ago
- A very simple cross-service LLM API for Python☆22Updated 2 years ago
- GPU prices aggregator for cloud providers☆44Updated last week
- TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces☆141Updated last year
- ☆74Updated 2 years ago
- Command line tool for Deep Infra cloud ML inference service☆33Updated last year
- Generates grammer files from typescript for LLM generation☆38Updated last year
- EXO Gym is an open-source Python toolkit that facilitates distributed AI research.☆87Updated 2 weeks ago
- Approximation of the Claude 3 tokenizer by inspecting generation stream☆148Updated last year