Mattbusel / tokio-prompt-orchestratorView on GitHub
Tokio pipeline for serving LLM requests: bounded five-stage flow with dedup, circuit breakers, retries, rate limits and a dead-letter queue in front of Anthropic, OpenAI, llama.cpp or vLLM. REST, SSE, MCP, TUI.
☆59Oct 4, 2026Updated this week

Alternatives and similar repositories for tokio-prompt-orchestrator

Users that are interested in tokio-prompt-orchestrator are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

Sorting:

Are these results useful?