ubergarm / r1-ktransformers-guideLinks
run DeepSeek-R1 GGUFs on KTransformers
☆236Updated 3 months ago
Alternatives and similar repositories for r1-ktransformers-guide
Users that are interested in r1-ktransformers-guide are comparing it to the libraries listed below
Sorting:
- LM inference server implementation based on *.cpp.☆226Updated this week
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆256Updated 3 weeks ago
- High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.☆1,147Updated last week
- Community maintained hardware plugin for vLLM on Ascend☆791Updated this week
- gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。☆194Updated this week
- Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…☆633Updated this week
- LLM Inference benchmark☆421Updated 11 months ago
- A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations☆40Updated last month
- CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…☆151Updated this week
- Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch☆377Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆51Updated 8 months ago
- FlagScale is a large model toolkit based on open-sourced projects.☆307Updated this week
- vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60☆79Updated 2 weeks ago
- LLM 并发性能测试工具,支持自动化压力测试和性能报告生成。☆93Updated 3 months ago
- GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation☆205Updated last week
- vLLM Documentation in Chinese Simplified / vLLM 中文文档☆80Updated last month
- A fast communication-overlapping library for tensor/expert parallelism on GPUs.☆986Updated 3 weeks ago
- ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents☆500Updated 2 weeks ago
- Mixture-of-Experts (MoE) Language Model☆189Updated 9 months ago
- A demo built on Megrez-3B-Instruct, integrating a web search tool to enhance the model's question-and-answer capabilities.☆38Updated 6 months ago
- Low-bit LLM inference on CPU/NPU with lookup table☆811Updated 3 weeks ago
- GLM Series Edge Models☆142Updated 2 weeks ago
- A streamlined and customizable framework for efficient large model evaluation and performance benchmarking☆1,203Updated this week
- 中文Mixtral混合专家大模型(Chinese Mixtral MoE LLMs)☆603Updated last year
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆129Updated 2 weeks ago
- RAG SYSTEM FOR RWKV☆49Updated 6 months ago
- xllamacpp - a Python wrapper of llama.cpp☆44Updated last week
- Review/Check GGUF files and estimate the memory usage and maximum tokens per second.☆177Updated last week
- a huggingface mirror site.☆288Updated last year
- llama.cpp fork with additional SOTA quants and improved performance☆608Updated this week