An OpenAI API compatible LLM inference server based on ExLlamaV2.
☆24Feb 9, 2024Updated 2 years ago
Alternatives and similar repositories for exllamav2-openai-server
Users that are interested in exllamav2-openai-server are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Train Llama Loras Easily☆30Aug 3, 2023Updated 3 years ago
- The world's first pronounceable programming notation.☆11Dec 11, 2023Updated 2 years ago
- A fast batching API to serve LLM models☆188Apr 26, 2024Updated 2 years ago
- Multi-Domain Expert Learning☆65Jan 23, 2024Updated 2 years ago
- Reinforcement learning algorithms implemented to learn to play Super Mario Bros.☆12Oct 24, 2020Updated 6 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Extract a single expert from a Mixture Of Experts model using slerp interpolation.☆19May 26, 2024Updated 2 years ago
- Easily convert HuggingFace models to GGUF-format for llama.cpp☆21Jul 27, 2024Updated 2 years ago
- Simple node proxy for llama-server that enables MCP use☆19May 10, 2025Updated last year
- A free AI text generation interface based on KoboldAI☆33Feb 27, 2024Updated 2 years ago
- Eh, simple and works.☆27Dec 9, 2023Updated 2 years ago
- ☆14Oct 31, 2023Updated 3 years ago
- ☆28Aug 30, 2023Updated 3 years ago
- ☆27Sep 29, 2026Updated last month
- [TMLR] "Can You Win Everything with Lottery Ticket?" by Tianlong Chen, Zhenyu Zhang, Jun Wu, Randy Huang, Sijia Liu, Shiyu Chang, Zhangy…☆10Sep 8, 2022Updated 4 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- This is my personal nextJS tool for sandboxing/testing GPT 3 prompts.☆12Jan 29, 2026Updated 9 months ago
- Convert PyTorch checkpoints, safetensors files and whole Hugging Face model folders to fp16, bf16, fp8 or ComfyUI fp8_scaled, int8_convro…☆12Oct 29, 2026Updated last week
- A program to automate testing open source LLMs for their political compass scores☆11Aug 24, 2026Updated 2 months ago
- A single notebook for fine-tuning GPT-3.5 turbo☆31Aug 16, 2024Updated 2 years ago
- Conversion script adapting vicuna dataset into alpaca format for use with oobabooga's trainer☆12Jun 21, 2023Updated 3 years ago
- A MCP server for secure Python code execution with artifact capture, virtual environment support, and LM Studio integration☆11Mar 14, 2026Updated 7 months ago
- A dashboard for the real-time monitoring of publicly available online COVID-19 posts across Facebook, Twitter, and Instagram by health-fo…☆13Sep 19, 2023Updated 3 years ago
- A free web-based procedural texture and volume generation toolkit for volumetric rendering, terrain generation.☆18Jun 23, 2026Updated 4 months ago
- Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…☆46Mar 21, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Automatically enables TensorFloat-32 (TF32) acceleration for NVIDIA RTX 30/40/50 series GPUs in ComfyUI.☆12Mar 10, 2026Updated 7 months ago
- About my PC setup and my scripts to automate workstation and server setup after a fresh OS install.☆16Dec 18, 2025Updated 10 months ago
- The official API server for Exllama. OAI compatible, lightweight, and fast.☆1,510Updated this week
- Professional video save node for ComfyUI. ProRes ladder, All-Intra H.264/H.265, FFV1 lossless. Honest 10-bit pipeline that preserves diff…☆10May 8, 2026Updated 5 months ago
- Extend the Conditioning of Stable Diffusion to take Audio Embeddings Instead of Text Embeddings using Wav2Vec2-BERT model☆13Sep 25, 2024Updated 2 years ago
- Improve Devcontainer Creation☆17Oct 29, 2026Updated last week
- Inference Llama 2 in one file of pure Haskell (A port of llama2.c from Andrej Karpathy)☆14Oct 17, 2025Updated last year
- Parametric 3D CAD built on FreeCAD: a live renderer, a kernel that finishes the job, and a model with a memory.☆21Updated this week
- OMARG-AIR/AID (Our Models and Applications Repository for General Artificial Intelligence Research and AI Development) - A hub for curate…☆14Feb 26, 2026Updated 8 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Skype Code Samples, including ChatGPT, GPT3.5, GPT-4 powered bots☆13May 10, 2024Updated 2 years ago
- Base Gradient Container☆18Feb 2, 2024Updated 2 years ago
- ☆35Dec 15, 2025Updated 10 months ago
- ☆13Sep 16, 2026Updated last month
- ☆16Nov 25, 2023Updated 2 years ago
- ☆12Oct 15, 2022Updated 4 years ago
- Large-scale LLM inference engine☆1,903Updated this week