A high performance batching router optimises max throughput for text inference workload
☆16Sep 6, 2023Updated 3 years ago
Alternatives and similar repositories for text-inference-batcher
Users that are interested in text-inference-batcher are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- fused MoE kernel in Candle backend☆14Feb 3, 2026Updated 9 months ago
- Using modal.com to process FineWeb-edu data☆21Sep 12, 2026Updated last month
- the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly☆33Oct 19, 2024Updated 2 years ago
- Graph model execution API for Candle☆19Jul 27, 2025Updated last year
- 基于 CUDA Driver API 的 cuda 运行时环境☆16Jul 30, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- This library supports evaluating disparities in generated image quality, diversity, and consistency between geographic regions.☆20Jun 3, 2024Updated 2 years ago
- Unleash the power of FOSS language models on your local machine☆30Nov 21, 2023Updated 2 years ago
- LLM inference optimization simulator☆13Jul 12, 2025Updated last year
- Experimental Pyodide fork which works in Cloudflare Workers☆15Dec 7, 2022Updated 3 years ago
- A Scheduler for Batched LLM Inference☆19Oct 5, 2025Updated last year
- Yet another frontend for LLM, written using .NET and WinUI 3☆11Sep 14, 2025Updated last year
- Multi-dataset evaluation of Anthropic's J-Space on Qwen3-4B. Demonstrates that internal 'workspace noise' can reliably route high-confide…☆12Jul 13, 2026Updated 3 months ago
- Simple Tool Caller for llama.cpp☆11Aug 12, 2024Updated 2 years ago
- Remote control agent for macOS development☆18Sep 20, 2026Updated last month
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- B-Llama3o a llama3 with Vision Audio and Audio understanding as well as text and Audio and Animation Data output.☆26Jun 3, 2024Updated 2 years ago
- ☆14Jun 28, 2023Updated 3 years ago
- ☆16Apr 13, 2024Updated 2 years ago
- The main HSIMP file☆11Jul 28, 2017Updated 9 years ago
- You can use it to modify HTTP (S) response values, redirect static file requests to the local file directory, and support batch modificat…☆18Nov 30, 2022Updated 3 years ago
- Jupyter Notebooks and other code for 4CE data visualizations.☆14Jan 25, 2023Updated 3 years ago
- ☆14Oct 31, 2023Updated 3 years ago
- MCP server that provides a service for graph-ify-ing the codebase then retrieving from it to help feed perfect amount of context.☆11Aug 6, 2025Updated last year
- Demo code for Gemini Live Integration☆13Jul 29, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- a SplineCamera react component☆14Feb 18, 2024Updated 2 years ago
- PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation☆33Nov 16, 2024Updated last year
- Attach to a process and modify environment on exec☆11Apr 5, 2013Updated 13 years ago
- ☆13Jun 25, 2016Updated 10 years ago
- Our winning entry to AWS Serverless Chatbot Hackathon. A Slack bot built using AWS API Gateway & Lambda.☆13Oct 17, 2016Updated 10 years ago
- ☆14Jan 25, 2024Updated 2 years ago
- ☆10May 28, 2025Updated last year
- Experimentation in decoding serial UART for the ES200G battery—only the battery, not the scooter's communications to it.☆12Oct 30, 2022Updated 4 years ago
- [EMNLP 24] Source code for paper 'AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tu…☆13Dec 15, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- OCI images for the curious.☆12Mar 21, 2022Updated 4 years ago
- A simple speech-to-text and text-to-speech AI chatbot that can be run fully offline.☆46Jan 28, 2024Updated 2 years ago
- Diffusion_TTS extension for booga☆71Sep 6, 2025Updated last year
- An experiment to see if we can combine Plain Text Accounting with TigerBeetle☆10Nov 8, 2022Updated 3 years ago
- TLS-terminating reverse proxy in Go☆14Updated this week
- Program used by developers to create a graphical interface .ui file☆19Jan 27, 2026Updated 9 months ago
- Course Project for COMP4471 on RWKV☆17Feb 11, 2024Updated 2 years ago