A high performance batching router optimises max throughput for text inference workload
☆16Sep 6, 2023Updated 2 years ago
Alternatives and similar repositories for text-inference-batcher
Users that are interested in text-inference-batcher are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Using modal.com to process FineWeb-edu data☆20Aug 25, 2026Updated last week
- the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly☆33Oct 19, 2024Updated last year
- 🚀 LLM inference optimization simulator, modeling compute-bound prefill and memory-bound decode phases.☆13Jul 12, 2025Updated last year
- Yet another frontend for LLM, written using .NET and WinUI 3☆11Sep 14, 2025Updated 11 months ago
- UI for Ollama☆14Aug 21, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 🔌 Check if a website is up with Deno☆14Jun 1, 2020Updated 6 years ago
- This library supports evaluating disparities in generated image quality, diversity, and consistency between geographic regions.☆20Jun 3, 2024Updated 2 years ago
- Unleash the power of FOSS language models on your local machine☆30Nov 21, 2023Updated 2 years ago
- Sparse autoencoders for Contra text embedding models☆25Apr 24, 2024Updated 2 years ago
- Experimental Pyodide fork which works in Cloudflare Workers☆16Dec 7, 2022Updated 3 years ago
- a SplineCamera react component☆14Feb 18, 2024Updated 2 years ago
- Simple Tool Caller for llama.cpp☆11Aug 12, 2024Updated 2 years ago
- PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation☆33Nov 16, 2024Updated last year
- Geospatial Next.js app with DuckDB-Wasm☆15May 24, 2023Updated 3 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Demo code for Gemini Live Integration☆13Jul 29, 2025Updated last year
- The main HSIMP file☆11Jul 28, 2017Updated 9 years ago
- ☆14Oct 31, 2023Updated 2 years ago
- ComfyUI node to beautify your image generation prompt☆18Sep 10, 2025Updated 11 months ago
- Diffusion_TTS extension for booga☆71Sep 6, 2025Updated 11 months ago
- A simple speech-to-text and text-to-speech AI chatbot that can be run fully offline.☆45Jan 28, 2024Updated 2 years ago
- Pynocular is a lightweight ORM that lets you query your database using Pydantic models and asyncio☆11May 24, 2022Updated 4 years ago
- Apps that run on modal.com☆13Sep 14, 2025Updated 11 months ago
- Synthèses vocale piper oobabooga☆14Feb 24, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Typed REST queries - build, transport, validate, execute. The packages implement a JSON-API-style query scheme (fields, filters, relation…☆33Updated this week
- A template for apps built with TinyBase, TypeScript, React, and ElectricSQL.☆13Feb 10, 2026Updated 6 months ago
- ☆14Jan 25, 2024Updated 2 years ago
- Our winning entry to AWS Serverless Chatbot Hackathon. A Slack bot built using AWS API Gateway & Lambda.☆13Oct 17, 2016Updated 9 years ago
- A CrowdSec Bouncer that syncs the decisions made by CrowdSec with CloudFlare's firewall using cloudflare workers. Manages multi user, mul…☆18Aug 17, 2026Updated 2 weeks ago
- utilities for loading and running text embeddings with onnx☆46Aug 16, 2025Updated last year
- This is an LLM interface that you can use to analyze and get insight into diary entries or other documents completely offline.☆16Dec 31, 2023Updated 2 years ago
- Training code for Sparse Autoencoders on Embedding models☆40Jul 11, 2026Updated last month
- Categorize credit card transactions using a local large language model similar to GPT3☆15Dec 29, 2023Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A repository to store helpful information and emerging insights in regard to LLMs☆21Oct 27, 2023Updated 2 years ago
- Perform facts checks on your conversations with LLMs to catch fake-news, misleading information, and LLMs confusion.☆10Apr 22, 2023Updated 3 years ago
- Guide to deploying Slurm and OpenMPI on Raspberry Pi computers☆13Nov 9, 2024Updated last year
- CoronaWhy Common Research and Data Infrastructure for COVID-19☆13Dec 2, 2020Updated 5 years ago
- That is an AWS CDK custom construct based on Tony's amazing Prowler Security, Hardening, Best Practises Tool https://github.com/toniblyx/…☆24May 10, 2023Updated 3 years ago
- Tiny Go libraries that I keep copying between projects☆12Updated this week
- Sync Taskwarrior tasks with Habitica☆15Sep 14, 2022Updated 3 years ago