wangcx18 / llm-vscode-inference-serverLinks
An endpoint server for efficiently serving quantized open-source LLMs for code.
☆58Updated 2 years ago
Alternatives and similar repositories for llm-vscode-inference-server
Users that are interested in llm-vscode-inference-server are comparing it to the libraries listed below
Sorting:
- starcoder server for huggingface-vscdoe custom endpoint☆179Updated 2 years ago
- Some simple scripts that I use day-to-day when working with LLMs and Huggingface Hub☆161Updated 2 years ago
- ☆164Updated 4 months ago
- ☆198Updated last year
- Visual Studio Code extension for WizardCoder☆149Updated 2 years ago
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆64Updated 2 years ago
- An OpenAI Completions API compatible server for NLP transformers models☆65Updated 2 years ago
- ☆68Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆53Updated 2 years ago
- Deploy your GGML models to HuggingFace Spaces with Docker and gradio☆38Updated 2 years ago
- ☆54Updated 2 years ago
- Client-side toolkit for using large language models, including where self-hosted☆113Updated 3 weeks ago
- A fast batching API to serve LLM models☆189Updated last year
- LLaMA Server combines the power of LLaMA C++ with the beauty of Chatbot UI.☆130Updated 2 years ago
- ☆117Updated last year
- Python client library for improving your LLM app accuracy☆97Updated 10 months ago
- Gradio based tool to run opensource LLM models directly from Huggingface☆96Updated last year
- ☆134Updated 2 weeks ago
- An OpenAI-like LLaMA inference API☆113Updated 2 years ago
- OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)☆275Updated 2 years ago
- Examples of models deployable with Truss☆211Updated this week
- Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniqu…☆76Updated last year
- Locally running LLM with internet access☆97Updated 5 months ago
- ⚡️🧪 Fast LLM Tool Calling Experimentation, big and smol☆152Updated last year
- Pipeline is an open source python SDK for building AI/ML workflows☆138Updated last year
- This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows inste…☆126Updated last year
- ☆66Updated 8 months ago
- run ollama & gguf easily with a single command☆52Updated last year
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆124Updated 2 years ago
- ezlocalai is an easy to set up local artificial intelligence server with OpenAI Style Endpoints.☆89Updated this week