Alternatives and similar repositories for ktrun

Users that are interested in ktrun are comparing it to the libraries listed below

• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆251Updated 6 months ago
• nlzy / vllm-gfx906
  vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60
  ☆247Updated this week
• thu-pacman / chitu
  High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
  ☆1,278Updated this week
• gengchaogit / triton-3.2.0-mi50
  triton3.2.0添加mi25/mi50/mi60支持
  ☆14Updated 4 months ago
• fattank / deepseek-on-vllm-VRAMandDRAM
  一套基于Vllm的显存内存混合模式大模型部署工具（图形界面），VRAMandDRAM模式虽然慢一点，但是解决了超大模型在普通家用计算机上的部署问题。
  ☆85Updated 4 months ago
• liuwenzhoa / KT_Qwen3
  ☆35Updated 4 months ago
• zj591227045 / DeepStressModel
  ☆166Updated 5 months ago
• vproxy-tools / ktransformers
  A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
  ☆43Updated 4 months ago
• lework / llm-benchmark
  LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。
  ☆159Updated 5 months ago
• vllm-project / vllm-ascend
  Community maintained hardware plugin for vLLM on Ascend
  ☆1,128Updated this week
• TylunasLi / fastllm
  纯c++的全平台llm加速库，支持python调用，支持chatglm-6B, llama, baichuan, moss基座，x86 / ARM
  ☆13Updated last week
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。
  ☆209Updated this week
• KylinMountain / graphrag-server
  添加🚀流式 Web 服务到 GraphRAG，兼容 OpenAI SDK，支持可访问的实体链接🔗，支持建议问题，兼容本地嵌入模型，修复诸多问题。Add streaming web server to GraphRAG, compatible with OpenAI SD…
  ☆261Updated 5 months ago
• MooreThreads / vllm_musa
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆62Updated 10 months ago
• MooreThreads / torch_musa
  torch_musa is an open source repository based on PyTorch, which can make full use of the super computing power of MooreThreads graphics c…
  ☆432Updated this week
• sophgo / LLM-TPU
  Run generative AI models in sophgo BM1684X/BM1688
  ☆240Updated last week
• weizxfree / KnowFlow
  KnowFlowRAG
  ☆287Updated this week
• llama-factory / llamafactory
  llamafactory blog
  ☆40Updated 11 months ago
• zstar1003 / ragflow-plus
  Ragflow-Plus 是 Ragflow 的二次开发版本，使其更为简洁实用
  ☆1,015Updated 2 weeks ago
• RapidAI / LLM-EXAM
  大模型中文测试题库-民间版本
  ☆88Updated 2 years ago
• brightwang / graphrag-dify
  ☆265Updated 8 months ago
• NanGePlus / KagTest
  KAG开源框架介绍及使用KAG实现知识增强生成应用(产品模式测试、开发者模式测试)，KAG是OpenSPG发布v0.5版本中推出的知识增强生成（KAG）的专业领域知识服务框架，旨在充分利用知识图谱和向量检索的优势，增强大型语言模型和知识图谱，以解决 RAG 挑战
  ☆144Updated 5 months ago
• swordswind / ai_virtual_mate_linux
  AI虚拟伙伴Linux版
  ☆109Updated last month
• AdamPlatin123 / Open-Deep-Research-workflow-on-Dify
  ☆287Updated 7 months ago
• junjiem / dify-plugin-repackaging
  Scripting tool for downloading Dify plugin package from Dify Marketplace and Github and repackaging [true] offline package.
  ☆447Updated 2 weeks ago
• MegEngine / InferLLM
  a lightweight LLM model inference framework
  ☆739Updated last year
• modelscope / evalscope
  A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
  ☆1,690Updated this week
• William-GuoWei / ProposalLLM
  标书大模型（Proposal-LLM Chinese version )
  ☆271Updated 10 months ago
• UnicomAI / Unichat-llama3-Chinese
  ☆351Updated last year
• AIIRWKV / RWKV-RAG
  RAG SYSTEM FOR RWKV
  ☆51Updated 9 months ago