Alternatives and similar repositories for ktrun

Users that are interested in ktrun are comparing it to the libraries listed below

• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆252Updated 7 months ago
• nlzy / vllm-gfx906
  vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60
  ☆280Updated this week
• thu-pacman / chitu
  High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
  ☆1,293Updated this week
• fattank / deepseek-on-vllm-VRAMandDRAM
  一套基于Vllm的显存内存混合模式大模型部署工具（图形界面），VRAMandDRAM模式虽然慢一点，但是解决了超大模型在普通家用计算机上的部署问题。
  ☆86Updated 5 months ago
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆279Updated last month
• zj591227045 / DeepStressModel
  ☆168Updated 6 months ago
• lework / llm-benchmark
  LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
  ☆160Updated 6 months ago
• liuwenzhoa / KT_Qwen3
  ☆38Updated 5 months ago
• TylunasLi / fastllm
  纯c++的全平台llm加速库，支持python调用，支持chatglm-6B, llama, baichuan, moss基座，x86 / ARM
  ☆13Updated 3 weeks ago
• MooreThreads / vllm_musa
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆64Updated 11 months ago
• KylinMountain / graphrag-server
  添加🚀流式 Web 服务到 GraphRAG，兼容 OpenAI SDK，支持可访问的实体链接🔗，支持建议问题，兼容本地嵌入模型，修复诸多问题。Add streaming web server to GraphRAG, compatible with OpenAI SD…
  ☆260Updated 6 months ago
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。
  ☆213Updated 2 weeks ago
• MooreThreads / torch_musa
  torch_musa is an open source repository based on PyTorch, which can make full use of the super computing power of MooreThreads graphics c…
  ☆435Updated 3 weeks ago
• vllm-project / vllm-ascend
  Community maintained hardware plugin for vLLM on Ascend
  ☆1,179Updated last week
• modelscope / evalscope
  A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
  ☆1,762Updated last week
• Ikaros-521 / GraphRAG-Ollama-UI
  GraphRAG-Ollama-UI + GraphRAG4OpenWebUI 融合版（有gradio webui配置生成RAG索引，有fastapi提供RAG API服务）
  ☆104Updated last year
• adoresever / Pretuning
  A tool for creating pre-training datasets for language models, supporting one-click batch processing for both text and image datasets. 一个…
  ☆39Updated 9 months ago
• vproxy-tools / ktransformers
  A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
  ☆43Updated 5 months ago
• gpustack / gpustack
  Simple, scalable AI model deployment on GPU clusters
  ☆3,799Updated last week
• brightwang / graphrag-dify
  ☆269Updated 9 months ago
• RapidAI / LLM-EXAM
  大模型中文测试题库-民间版本
  ☆89Updated 2 years ago
• padeoe / hf-mirror-site
  a huggingface mirror site.
  ☆304Updated last year
• terryaic / OpenKAG
  OpenKAG (Open Knowledge Augmented Generation), is an enterprise intelligent knowledge platform based on large model technology.
  ☆55Updated 6 months ago
• hustyichi / dify-rag
  dify's rag patch module
  ☆275Updated last month
• ShaohonChen / transformers_from_scratch
  pretrain a wiki llm using transformers
  ☆51Updated last year
• LuckLittleBoy / SenseVoice-OneApi
  基于SenseVoice的funasr版本进行的api发布，可以无缝对接oneapi
  ☆75Updated last year
• luyilong2015 / open-webui-pipeline-for-ragflow
  使用open-webui中的pipelines技术在open-webui中调用ragflow的agent实现基于知识库的智能对话，并拥有美观的界面。
  ☆132Updated 3 months ago
• sophgo / ChatGLM3-TPU
  run chatglm3-6b in BM1684X
  ☆40Updated last year
• XShengTech / MEGREZ
  🌈 MEGREZ | 🍒 Make Extendable GPU Resource EASY
  ☆116Updated last week
• jianggujin / ollama-desktop
  Ollama Desktop是基于Ollama引擎的一个桌面应用解决方案，用于在macOS、Windows和Linux操作系统上运行和管理Ollama模型的GUI工具。
  ☆172Updated 3 months ago