lucasjinreal / Crane
A Pure Rust based LLM (Any LLM based MLLM such as Spark-TTS) Inference Engine, powering by Candle framework.
☆85Updated this week
Alternatives and similar repositories for Crane:
Users that are interested in Crane are comparing it to the libraries listed below
- High-performance Text-to-Speech server with OpenAI-compatible API, 8 voices, emotion tags, and modern web UI. Optimized for RTX GPUs.☆160Updated this week
- ☆52Updated this week
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆73Updated 3 months ago
- idea: https://github.com/nyxkrage/ebook-groupchat/☆86Updated 7 months ago
- A pipeline parallel training script for LLMs.☆136Updated last week
- Open source LLM UI, compatible with all local LLM providers.☆173Updated 6 months ago
- ☆66Updated last month
- automatically quant GGUF models☆164Updated last week
- Orpheus Chat WebUI☆32Updated this week
- Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on differe…☆55Updated last month
- A Conversational Speech Generation Model with Gradio UI and OpenAI compatible API. UI and API support CUDA, MLX and CPU devices.☆143Updated this week
- ☆24Updated 2 months ago
- cli tool to quantize gguf, gptq, awq, hqq and exl2 models☆69Updated 3 months ago
- Guaranteed structured outputs from any language model. Eliminate 100% of schema violations and state tracking failures in your LLM applic…☆120Updated this week
- A real-time speech-to-speech chatbot powered by Whisper Small, Llama 3.2, and Kokoro-82M.☆196Updated 2 months ago
- A frontend for creative writing with LLMs☆122Updated 8 months ago
- AI management tool☆113Updated 4 months ago
- Fast Streaming TTS with Orpheus + WebRTC (with FastRTC)☆97Updated this week
- The hearth of The Pulsar App, fast, secure and shared inference with modern UI☆56Updated 3 months ago
- Easily view and modify JSON datasets for large language models☆71Updated 3 weeks ago
- ☆197Updated 2 weeks ago
- Realtime tts reading of large textfiles by your favourite voice. +Translation via LLM (Python script)☆52Updated 5 months ago
- Something similar to Apple Intelligence?☆59Updated 8 months ago
- ☆125Updated this week
- fast state-of-the-art speech models and a runtime that runs anywhere 💥☆55Updated last month
- ☆83Updated 3 months ago
- Blue-text Bot AI. Uses Ollama + AppleScript☆50Updated 10 months ago
- This project is a reverse-engineered version of Figma's tone changer. It uses Groq's Llama-3-8b for high-speed inference and to adjust th…☆88Updated 8 months ago
- klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs☆71Updated 6 months ago
- Moxin is a family of fully open-source and reproducible LLMs☆85Updated 2 weeks ago