arlo-phoenix / CTranslate2-rocmLinks
Fast inference engine for Transformer models
☆54Updated last year
Alternatives and similar repositories for CTranslate2-rocm
Users that are interested in CTranslate2-rocm are comparing it to the libraries listed below
Sorting:
- ☆422Updated 9 months ago
- Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.☆270Updated last week
- AMD (Radeon GPU) ROCm based setup for popular AI tools on Ubuntu 24.04.1☆216Updated last month
- Input text from speech in any Linux window, the lean, fast and accurate way, using whisper.cpp OFFLINE. Speak with local LLMs via llama.c…☆155Updated 5 months ago
- A daemon that automatically manages the performance states of NVIDIA GPUs.☆104Updated 2 months ago
- AI Inferencing at the Edge. A simple one-file way to run various GGML models with KoboldAI's UI with AMD ROCm offloading☆725Updated last week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆616Updated this week
- A complete package that provides you with all the components needed to get started of dive deeper into Machine Learning Workloads on Cons…☆44Updated 2 months ago
- ☆236Updated 2 years ago
- Prometheus exporter for Linux based GDDR6/GDDR6X VRAM and GPU Core Hot spot temperature reader for NVIDIA RTX 3000/4000 series GPUs.☆24Updated last year
- Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc☆2,147Updated last week
- The official API server for Exllama. OAI compatible, lightweight, and fast.☆1,109Updated 3 weeks ago
- ROCm docker images with fixes/support for extra architectures, such as gfx803/gfx1010.☆31Updated 2 years ago
- LLM Frontend in a single html file☆681Updated 2 weeks ago
- Whisper command line client compatible with original OpenAI client based on CTranslate2.☆1,183Updated 3 weeks ago
- Fork of ollama for vulkan support☆108Updated 10 months ago
- Make PyTorch models at least run on APUs.☆56Updated 2 years ago
- ☆63Updated 7 months ago
- ☆87Updated last month
- ROCm docker images with fixes/support for legecy architecture gfx803. eg.Radeon RX 590/RX 580/RX 570/RX 480☆81Updated 7 months ago
- ☆48Updated 2 years ago
- Stable Diffusion Docker image preconfigured for usage with AMD Radeon cards☆143Updated last year
- A utility that uses Whisper to transcribe videos and various translation APIs to translate the transcribed text and save them as SRT (sub…☆74Updated last year
- General Site for the GFX803 ROCm Stuff☆136Updated 4 months ago
- Web UI for ExLlamaV2☆514Updated 11 months ago
- The main repository for building Pascal-compatible versions of ML applications and libraries.☆160Updated 4 months ago
- Scripts to control NVIDIA GPUs using NVML API☆36Updated last year
- Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input your VRAM and RAM and the toolcha…☆76Updated this week
- llama-swap + a minimal ollama compatible api☆41Updated last week
- An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.☆841Updated 11 months ago