Local LLM Inference Speed Test Tool
☆116Jul 2, 2026Updated 2 weeks ago
Alternatives and similar repositories for llm_speedtest
Users that are interested in llm_speedtest are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- KTransformers 一键部署脚本☆59Apr 18, 2025Updated last year
- forked from vllm-project/flash-attention☆58May 9, 2026Updated 2 months ago
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆386Updated this week
- The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with 100+ tok/s single-request decode…☆431Jul 13, 2026Updated last week
- 大语言模型工具集☆28Aug 1, 2025Updated 11 months ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- A fork of vLLM enabling Pascal architecture GPUs☆34Feb 21, 2025Updated last year
- TensorRT depth-anything for anyone and anywhere☆16Jan 29, 2024Updated 2 years ago
- Run DeepSeek-V4-Flash on SM89 (Ada / RTX 4090) with vLLM — patch over PR #41834. Validated on 4x RTX 4090.☆22Updated this week
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆30Jan 22, 2026Updated 5 months ago
- Patched NVIDIA GRID Drivers☆36May 1, 2026Updated 2 months ago
- ⚡️Qwen-Image 4.8x🎉 speedup with Hybrid Acceleration for low VRAM GPUs☆17Oct 24, 2025Updated 8 months ago
- Flash Attention in ~100 lines of CUDA (forward pass only)☆12Jun 10, 2024Updated 2 years ago
- 华为软件大赛： 最小费用流(zkw)+遗传算法。 zkw比sfa的费用流快好多啊，10倍左右，牛逼~☆13Apr 6, 2017Updated 9 years ago
- paper-read-notes☆13Sep 26, 2024Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- LLM Inference FrameWork☆30May 20, 2026Updated 2 months ago
- ncnn 实现一些项目例子☆27Feb 17, 2023Updated 3 years ago
- 搜藏的希望的代码片段☆13Jun 6, 2023Updated 3 years ago
- HunyuanDiT with TensorRT and libtorch☆18May 22, 2024Updated 2 years ago
- 使用mnn-llm对GOT-OCR2.0进行推理☆14Oct 2, 2024Updated last year
- The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…☆90Mar 25, 2026Updated 3 months ago
- ☆17Nov 14, 2023Updated 2 years ago
- 跟着Tensorrt_pro学习各种知识☆39Nov 25, 2022Updated 3 years ago
- A lightweight, single-header C++11 Jinja2 template engine for LLM chat templates.☆20Mar 4, 2026Updated 4 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Inference Llama 2 in one file of pure Cuda☆17Aug 20, 2023Updated 2 years ago
- Detection and Tracking ROS node based on CenterPoint and Kalman Filter☆24Feb 24, 2024Updated 2 years ago
- 一个轻量化的大模型推理框架☆23May 26, 2025Updated last year
- GEMV implementation with CUTLASS☆21Aug 21, 2025Updated 11 months ago
- A curated collection of reusable AI Agent Skills for standardized workflows, best practices, and domain expertise.☆22May 29, 2026Updated last month
- A ESP32 BLE scanner with iotWebConf and MQTT☆13Mar 30, 2019Updated 7 years ago
- Comfyui custom node for FunAudioLLM include CosyVoice2, SenseVoice and InspireMusic☆15Jan 9, 2026Updated 6 months ago
- ☆47Apr 16, 2026Updated 3 months ago
- 基于 C++23 的模块化 LLM 推理框架，原生解析 GGUF 格式。☆39Jun 19, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- FORK of VLLM for AMD MI25/50/60. A high-throughput and memory-efficient inference and serving engine for LLMs☆71May 4, 2025Updated last year
- ☆20Dec 29, 2023Updated 2 years ago
- A Minimalistic Auto-Diff Optimization Framework for Teaching and Understanding Pytorch☆27Jul 9, 2026Updated last week
- People and asset tracking system. Hardware support : 1. Happy Bubble Presence Detectors 2.Raspberryp Pi 3. ESP32 kit with Wifi and Blueto…☆10Jun 6, 2018Updated 8 years ago
- A tool for remote computer manager, support windows remote desktop and ssh.☆16Jun 19, 2025Updated last year
- 我的nonebot机器人☆15Updated this week
- ☆26Aug 15, 2023Updated 2 years ago