perk11 / large-model-proxy
Large Model Proxy is designed to make it easy to run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources. It listens on a dedicated port for each proxied LM, making them always available to the clients connecting to these ports.
☆47Updated last month
Related projects ⓘ
Alternatives and complementary repositories for large-model-proxy
- Easily view and modify JSON datasets for large language models☆62Updated last month
- An open source, Gradio-based chatbot app that combines the best of retrieval augmented generation and prompt engineering into an intellig…☆43Updated 3 months ago
- ☆25Updated last month
- A stock market bot that automatically, once a day, rebalances your Robinhood portfolio by gathering information about each ticker in the …☆34Updated 3 weeks ago
- CaSIL is an advanced natural language processing system that implements a sophisticated four-layer semantic analysis architecture. It pro…☆62Updated 2 weeks ago
- HTTP proxy for on-demand model loading with llama.cpp (or other OpenAI compatible backends)☆41Updated this week
- klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs☆58Updated last month
- ☆30Updated 6 months ago
- idea: https://github.com/nyxkrage/ebook-groupchat/☆82Updated 3 months ago
- ☆112Updated this week
- run ollama & gguf easily with a single command☆48Updated 6 months ago
- Experimental LLM Inference UX to aid in creative writing☆106Updated 4 months ago
- Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.☆25Updated this week
- A frontend for creative writing with LLMs☆108Updated 4 months ago
- Gradio based tool to run opensource LLM models directly from Huggingface☆87Updated 4 months ago
- SoftWhisper simplifies audio and video transcription using the powerful Whisper model. Easily select custom models, languages, and tasks,…☆33Updated last month
- The hearth of The Pulsar App, fast, secure and shared inference with modern UI☆35Updated 3 weeks ago
- An extension that lets the AI take the wheel, allowing it to use the mouse and keyboard, recognize UI elements, and prompt itself :3...no…☆96Updated 3 weeks ago
- After my server ui improvements were successfully merged, consider this repo a playground for experimenting, tinkering and hacking around…☆56Updated 3 months ago
- ☆128Updated this week
- Realtime tts reading of large textfiles by your favourite voice. +Translation via LLM (Python script)☆47Updated last month
- ☆104Updated 8 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆55Updated last week
- LLM backed Fantasy Tribe Game☆18Updated 2 weeks ago
- AI-Powered Web Automation Chat. This browser extension adds an AI chat widget to every page, allowing you to easily automate hundreds of …☆12Updated 3 weeks ago
- AnyModal is a Flexible Multimodal Language Model Framework☆40Updated this week
- 5X faster 60% less memory QLoRA finetuning☆21Updated 5 months ago
- ☆64Updated 5 months ago
- ☆18Updated 3 weeks ago