๐ LLM inference optimization simulator, modeling compute-bound prefill and memory-bound decode phases.
โ13Jul 12, 2025Updated 7 months ago
Alternatives and similar repositories for llm-inference-simulator
Users that are interested in llm-inference-simulator are comparing it to the libraries listed below
Sorting:
- Graph model execution API for Candleโ17Jul 27, 2025Updated 7 months ago
- A high performance batching router optimises max throughput for text inference workloadโ16Sep 6, 2023Updated 2 years ago
- ๅบไบ CUDA Driver API ็ cuda ่ฟ่กๆถ็ฏๅขโ15Jul 30, 2025Updated 7 months ago
- A curated list of awesome papers about utilizing large language models for ranking.โ31Oct 30, 2024Updated last year
- Log file scanner used with EDA tools to classify errors and warningsโ12Nov 14, 2022Updated 3 years ago
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.โ32Sep 19, 2025Updated 5 months ago
- โ11Apr 25, 2020Updated 5 years ago
- User-friendly viewer for Parquet filesโ10Jan 10, 2026Updated last month
- fine-tuning tutorialโ18Feb 20, 2026Updated last week
- DOS Program Developmentโ13Nov 9, 2022Updated 3 years ago
- A domain-specific language (DSL) based on Triton but providing higher-level abstractions.โ41Feb 4, 2026Updated last month
- A Vanilla Web Component wrapper for hCaptcha. Allows for easy integration with hCaptcha in many modern web frameworks.โ18Feb 20, 2026Updated last week
- Protocol buffers and other common resources.โ13Jan 20, 2026Updated last month
- A Redis-compatible in-memory database server written in Rust with MLua-based Lua 5.1 scriptingโ17Nov 28, 2025Updated 3 months ago
- Browser based ML Inference | OpenAI compliant | Run models like DeepSeek, Llama 3.2, NomicEmbed, KokoroTTS, and moreโ52Mar 4, 2025Updated last year
- This project is based on the [LTX-Video](https://github.com/Lightricks/LTX-Video) algorithm of the diffusers and optimized and accelerateโฆโ13Dec 31, 2024Updated last year
- A Library for Scaling Mixed-Integer Optimization-Based Machine Learning.โ12Jun 24, 2024Updated last year
- โ10Jan 9, 2024Updated 2 years ago
- Redis distributed lock implementation for Python based on Pub/Sub messagingโ11Feb 14, 2026Updated 2 weeks ago
- LightGBM for handling label-imbalanced data with focal and weighted loss functions in binary and multiclass classificationโ21Jan 29, 2026Updated last month
- โ14Dec 12, 2022Updated 3 years ago
- โ11Dec 6, 2023Updated 2 years ago
- Source code for LEF/DEFโ11Oct 16, 2018Updated 7 years ago
- A complete(grpc service and lib) Rust inference with multilingual embedding support. This version leverages the power of Rust for both GRโฆโ40Aug 20, 2024Updated last year
- An FPGA-accelerated platform for FEC analysis of wireline systemsโ12Jan 21, 2025Updated last year
- Less-Resilient MapReduce for Goโ10Feb 15, 2023Updated 3 years ago
- Clober Solidity Libraryโ10Jun 9, 2025Updated 8 months ago
- Demo of fine-tuning QA models for answering FAQ of cloud providers documentationโ11Mar 7, 2023Updated 2 years ago
- msx game development library ubox exampleโ11Apr 26, 2023Updated 2 years ago
- Sphinx extension for visual documentation of hardware written in HWTโ11Nov 12, 2025Updated 3 months ago
- example apps for inference.shโ20Updated this week
- โ17Jan 4, 2026Updated 2 months ago
- Hunt Town is a web3 co-building community where builders come together to contribute to the expansion of web3 culture and products.โ14Jan 15, 2026Updated last month
- Token-aware HTML chunking that preserves structure and attributes, with optional cleaning and attribute length control.โ15Aug 12, 2025Updated 6 months ago
- Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!โ10Aug 29, 2018Updated 7 years ago
- โ13Jan 7, 2025Updated last year
- BERT score for text generationโ12Jan 15, 2025Updated last year
- ๐น Instruct.KR 2025 Summer Meetup: ์คํ์์ค LLM, vLLM์ผ๋ก Production๊น์ง ๐นโ23Aug 2, 2025Updated 7 months ago
- This project auto-instruments containerized workloads in Kubernetes with New Relic agents.โ12Updated this week