intentee / paddlerLinks
Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙
☆1,379Updated last week
Alternatives and similar repositories for paddler
Users that are interested in paddler are comparing it to the libraries listed below
Sorting:
- Minimal LLM inference in Rust☆1,025Updated last year
- A high-performance inference engine for AI models☆1,383Updated this week
- Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.☆386Updated last year
- A cross-platform browser ML framework.☆725Updated last year
- VS Code extension for LLM-assisted code/text completion☆1,072Updated 2 weeks ago
- WebAssembly binding for llama.cpp - Enabling on-browser LLM inference☆942Updated last week
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆540Updated this week
- A realtime serving engine for Data-Intensive Generative AI Applications☆1,074Updated last week
- SeekStorm - sub-millisecond full-text search library & multi-tenancy server in Rust☆1,782Updated this week
- A multi-platform desktop application to evaluate and compare LLM models, written in Rust and React.☆878Updated 2 weeks ago
- Big & Small LLMs working together☆1,211Updated this week
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆239Updated 4 months ago
- Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc☆1,977Updated this week
- ♾️ Helix is a private GenAI stack for building AI agents with declarative pipelines, knowledge (RAG), API bindings, and first-class testi…☆545Updated last week
- Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.☆1,368Updated last week
- Blazingly fast LLM inference.☆6,253Updated last week
- ☆407Updated this week
- Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.☆2,755Updated last month
- Rust library for generating vector embeddings, reranking. Re-write of qdrant/fastembed.☆678Updated last week
- Fast, streaming indexing, query, and agentic LLM applications in Rust☆618Updated last week
- Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀☆856Updated 3 weeks ago
- Korvus is a search SDK that unifies the entire RAG pipeline in a single database query. Built on top of Postgres with bindings for Python…☆1,453Updated 10 months ago
- Super-fast Structured Outputs☆621Updated last week
- Embeddable library or single binary for indexing and searching 1B vectors☆337Updated this week
- Large-scale LLM inference engine☆1,600Updated last week
- Local voice chatbot for engaging conversations, powered by Ollama, Hugging Face Transformers, and Coqui TTS Toolkit☆785Updated last year
- git-like rag pipeline☆250Updated 2 weeks ago
- Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from R…☆519Updated last week
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆611Updated 9 months ago
- 🔥🔥 Kokoro in Rust. https://huggingface.co/hexgrad/Kokoro-82M Insanely fast, realtime TTS with high quality you ever have.☆639Updated last week