anarchy-ai / llm-speed-benchmark
Benchmarking tool for assessing LLM models' performance across different hardwares
☆16Updated last year
Alternatives and similar repositories for llm-speed-benchmark:
Users that are interested in llm-speed-benchmark are comparing it to the libraries listed below
- Public reports detailing responses to sets of prompts by Large Language Models.☆30Updated 2 months ago
- Transformer GPU VRAM estimator☆58Updated last year
- Runner in charge of collecting metrics from LLM inference endpoints for the Unify Hub☆17Updated last year
- An open source MCP proxy.☆9Updated 2 months ago
- A simple github actions script to build a llamafile and uploads to huggingface☆14Updated last year
- ☆7Updated 2 years ago
- Machine learning library, Distributed training, Deep learning, Reinforcement learning, Models, TensorFlow, PyTorch☆60Updated last week
- never forget anything again! combine AI and intelligent tooling for a local knowledge base to track catalogue, annotate, and plan for you…☆37Updated 10 months ago
- Python examples using the bigcode/tiny_starcoder_py 159M model to generate code☆44Updated last year
- ☆73Updated last year
- Tensor library for machine learning☆21Updated last year
- A better way of testing, inspecting, and analyzing AI Agent traces.☆30Updated this week
- LLM code editor for backend services☆14Updated 5 months ago
- A lightweight Python utility that aggregates and exports comprehensive system information to JSON, specifically designed for feeding syst…☆12Updated last month
- A benchmark framework for LLM serving performance, based on API call☆14Updated 11 months ago
- Kel - CLI-based AI assistant☆31Updated last year
- An endpoint server for efficiently serving quantized open-source LLMs for code.☆54Updated last year
- Tools for formatting large language model prompts.☆12Updated last year
- Web Interface for Vision Language Models Including InternVLM2☆20Updated 8 months ago
- Talk to GPT in Vim!☆16Updated 2 months ago
- Contains the model patches and the eval logs from the passing swe-bench-lite run.☆10Updated 9 months ago
- Docker images and configuration to run text-generation-webui with GPU or CPU support☆28Updated last year
- ☆17Updated last week
- Visualize expert firing frequencies across sentences in the Mixtral MoE model☆17Updated last year
- A collection of tools for your LLMs that run on Modal☆16Updated last month
- The AI-powered CLI Assistant☆26Updated 10 months ago
- Nexusflow function call, tool use, and agent benchmarks.☆19Updated 3 months ago
- The official Python library for Formulaic☆16Updated 11 months ago
- GraphRag vs Embeddings☆13Updated 8 months ago
- Falcon40B and 7B (Instruct) with streaming, top-k, and beam search☆40Updated last year