darkautism / llmserver-rsLinks
A Rust-based, OpenAI-style API server for large language models (LLMs)
☆13Updated this week
Alternatives and similar repositories for llmserver-rs
Users that are interested in llmserver-rs are comparing it to the libraries listed below
Sorting:
- Easy installation and usage of Rockchip's NPUs found in RK3588 and similar SoCs☆157Updated 3 weeks ago
- Easier usage of LLMs in Rockchip's NPU on SBCs like Orange Pi 5 and Radxa Rock 5 series☆137Updated 2 weeks ago
- Automated script to convert Huggingface and GGUF models to rkllm format for running on Rockchip NPU☆31Updated 7 months ago
- ☆49Updated 4 months ago
- Streaming TTS based on Piper with optional RK3588 NPU support☆93Updated 2 months ago
- top-like script for rockhip NPUs on linux☆44Updated last month
- Ollama alternative for Rockchip NPU: An efficient solution for running AI and Deep learning models on Rockchip devices with optimized NPU…☆216Updated this week
- Reverse engineering the rk3588 npu☆85Updated last year
- Run Large Language Models on RK3588 with GPU-acceleration☆107Updated last year
- Allows access via HTTP to LLM running on RK3588 NPU. Returns JSON response.☆25Updated last year
- Making Rockchip's RKNN-Toolkit-2 install easier for SBCs like Orange Pi 5 or Radxa Rock 5☆23Updated 3 weeks ago
- My develoopment fork of llama.cpp. For now working on RK3588 NPU and Tenstorrent backend☆97Updated last month
- ☆18Updated 5 months ago
- A build project for ONNX Runtime☆43Updated this week
- YoloV5 NPU for the RK3566/68/88☆103Updated last year
- ☆22Updated this week
- CGO bindings to RKNN-Toolkit2 to perform Inferencing in Go on Rockchip NPU☆103Updated this week
- 🖥️ myrktop - Orange Pi 5 (RK3588) System Monitoring script CPU,RAM,NPU,GPU,TEMPERATURES☆20Updated 2 months ago
- a cpp ggml port of "VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech." for use in mobile…☆41Updated 10 months ago
- Run your yolov7 object detection with Rockchip NPU platforms (RK3566, RK3568, RK3588, RK3588S, RV1103, RV1106, RK3562).☆55Updated last year
- Arch Linux ARM image builder and installer for aarch64 UEFI devices, focusing on rk3588☆66Updated 2 months ago
- ☆21Updated 4 months ago
- A ggml (C++) re-implementation of tortoise-tts☆187Updated 10 months ago
- Run stable-diffusion-webui with Radeon RX 580 8GB on Ubuntu 22.04.2 LTS☆64Updated last year
- ncnn benchmark on various single board computers☆160Updated last year
- Using FastChat-T5 Large Language Model, Vosk API for automatic speech recognition, and Piper for text-to-speech☆120Updated 2 years ago
- A Pure Rust based LLM (Any LLM based MLLM such as Spark-TTS) Inference Engine, powering by Candle framework.☆126Updated last week
- TTS support with GGML☆117Updated last week
- Windows on Arm drivers for RK35xx platforms.☆160Updated 3 months ago
- Mirrors of Rockchip BSP repositories, only contains yocto related ones to keep it thin since the yocto would try to clone the whole repos…☆56Updated last month