perk11 / large-model-proxy
Large Model Proxy is designed to make it easy to run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources. It listens on a dedicated port for each proxied LM, making them always available to the clients connecting to these ports.
☆49Updated 3 months ago
Alternatives and similar repositories for large-model-proxy:
Users that are interested in large-model-proxy are comparing it to the libraries listed below
- ☆27Updated 3 months ago
- Easily view and modify JSON datasets for large language models☆69Updated 3 months ago
- LLM backed Fantasy Tribe Game☆18Updated 2 months ago
- CaSIL is an advanced natural language processing system that implements a sophisticated four-layer semantic analysis architecture. It pro…☆64Updated 2 months ago
- ☆25Updated last week
- klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs☆67Updated 4 months ago
- Easy to use, High Performant Knowledge Distillation for LLMs☆40Updated 2 weeks ago
- Gradio based tool to run opensource LLM models directly from Huggingface☆90Updated 7 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆70Updated last month
- idea: https://github.com/nyxkrage/ebook-groupchat/☆85Updated 5 months ago
- SoftWhisper simplifies audio and video transcription using the powerful Whisper model. Easily select custom models, languages, and tasks,…☆41Updated 3 months ago
- ☆121Updated last week
- Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.☆24Updated last week
- A stock market bot that automatically, once a day, rebalances your Robinhood portfolio by gathering information about each ticker in the …☆49Updated last month
- After my server ui improvements were successfully merged, consider this repo a playground for experimenting, tinkering and hacking around…☆56Updated 5 months ago
- Something similar to Apple Intelligence?☆58Updated 6 months ago
- Who needs o1 anyways. Add CoT to any OpenAI compatible endpoint.☆41Updated 4 months ago
- SPLAA is an AI assistant framework that utilizes voice recognition, text-to-speech, and tool-calling capabilities to provide a conversati…☆23Updated 3 weeks ago
- "a towel is about the most massively useful thing an interstellar AI hitchhiker can have"☆48Updated 3 months ago
- ☆30Updated 8 months ago
- An open source, Gradio-based chatbot app that combines the best of retrieval augmented generation and prompt engineering into an intellig…☆48Updated 5 months ago
- The hearth of The Pulsar App, fast, secure and shared inference with modern UI☆53Updated last month
- SLOP Detector and analyzer based on dictionary for shareGPT JSON and text☆53Updated 2 months ago
- ☆16Updated last month
- This extension enhances the capabilities of textgen-webui by integrating advanced vision models, allowing users to have contextualized co…☆50Updated 3 months ago
- Text generation in Python, as easy as possible☆51Updated this week
- A frontend for creative writing with LLMs☆115Updated 6 months ago
- An extension that lets the AI take the wheel, allowing it to use the mouse and keyboard, recognize UI elements, and prompt itself :3...no…☆103Updated 3 months ago
- ☆27Updated 5 months ago
- This is the Mixture-of-Agents (MoA) concept, adapted from the original work by TogetherAI. My version is tailored for local model usage a…☆108Updated 7 months ago