manuelescobar-dev / LLM-ToolsLinks
Open-source calculator for LLM system requirements.
☆175Updated 11 months ago
Alternatives and similar repositories for LLM-Tools
Users that are interested in LLM-Tools are comparing it to the libraries listed below
Sorting:
- Comparison of Language Model Inference Engines☆235Updated 11 months ago
- Multi-Faceted AI Agent and Workflow Autotuning. Automatically optimizes LangChain, LangGraph, DSPy programs for better quality, lower exe…☆261Updated 6 months ago
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆249Updated last year
- ☆89Updated 11 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆708Updated this week
- ☆150Updated 5 months ago
- Modular and structured prompt caching for low-latency LLM inference☆103Updated last year
- A high-performance inference system for large language models, designed for production environments.☆482Updated 2 weeks ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆131Updated 2 months ago
- ArcticInference: vLLM plugin for high-throughput, low-latency inference☆300Updated this week
- The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Mem…☆388Updated last year
- A throughput-oriented high-performance serving framework for LLMs☆915Updated 3 weeks ago
- LLM Inference benchmark☆430Updated last year
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,152Updated last month
- ☆56Updated last year
- [ICML 2025 Oral] CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction☆560Updated 6 months ago
- Materials for learning SGLang☆650Updated this week
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆268Updated 3 months ago
- [NeurIPS 2025] Simple extension on vLLM to help you speed up reasoning model without training.☆206Updated 5 months ago
- Serverless LLM Serving for Everyone.☆603Updated this week
- LLM Serving Performance Evaluation Harness☆80Updated 8 months ago
- ☆431Updated 2 months ago
- LLMPerf is a library for validating and benchmarking LLMs☆1,046Updated 11 months ago
- ☆312Updated last week
- Benchmarking the serving capabilities of vLLM☆56Updated last year
- awesome llm plaza: daily tracking all sorts of awesome topics of llm, e.g. llm for coding, robotics, reasoning, multimod etc.☆210Updated 3 weeks ago
- ☆57Updated last year
- Benchmark suite for LLMs from Fireworks.ai☆83Updated last week
- [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration☆243Updated last year
- A list of LLM benchmark frameworks.☆72Updated last year