yigitkonur / rust-hyper-load-balanced-api-client

A high-performance Rust tool for sending API requests (to LLMs in my case) with built-in weighted load balancing, retry mechanisms, and rate limiting. Using hyper for fast request handling, it manages large volumes of asynchronous requests and is optimized for 10K request per second.
14Updated 5 months ago

Related projects

Alternatives and complementary repositories for rust-hyper-load-balanced-api-client