Alternatives and similar repositories for ktrun:

Users that are interested in ktrun are comparing it to the libraries listed below

• zj591227045 / DeepStressModel
  ☆124Updated last week
• thu-pacman / chitu
  High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
  ☆1,053Updated this week
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆212Updated 3 weeks ago
• win4r / GraphRAG4OpenWebUI
  GraphRAG4OpenWebUI integrates Microsoft's GraphRAG technology into Open WebUI, providing a versatile information retrieval API. It combin…
  ☆477Updated 2 months ago
• gpustack / gpustack
  Manage GPU clusters for running AI models
  ☆2,279Updated last week
• KylinMountain / graphrag-server
  添加🚀流式 Web 服务到 GraphRAG，兼容 OpenAI SDK，支持可访问的实体链接🔗，支持建议问题，兼容本地嵌入模型，修复诸多问题。Add streaming web server to GraphRAG, compatible with OpenAI SD…
  ☆242Updated this week
• kvcache-ai / ktransformers
  A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
  ☆13,218Updated this week
• 0x5446 / api4sensevoice
  API and websocket server for sensevoice. It has inherited some enhanced features, such as VAD detection, real-time streaming recognition,…
  ☆382Updated 5 months ago
• OpenBMB / UltraRAG
  Build & Optimize your RAG.
  ☆588Updated last week
• kvcache-ai / Mooncake
  Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
  ☆2,948Updated this week
• fufankeji / LLMs-Technology-Community-Beyondata
  《赋范大模型技术社区》是针对各阶大模型学习者量身打造的基于各类大模型，包括环境设置、本地部署、高效微调、开发实战等技能在内的全流程指导！
  ☆230Updated last month
• ymcui / Chinese-LLaMA-Alpaca-3
  中文羊驼大模型三期项目 (Chinese Llama-3 LLMs) developed from Meta Llama 3
  ☆1,900Updated 6 months ago
• Sumandora / remove-refusals-with-transformers
  Implements harmful/harmless refusal removal using pure HF Transformers
  ☆709Updated 9 months ago
• IEIT-Yuan / Yuan-2.0
  Yuan 2.0 Large Language Model
  ☆686Updated 8 months ago
• likelovewant / ROCmLibs-for-gfx1103-AMD780M-APU
  ROCm Library Files for gfx1103 and update with others arches based on AMD GPUs for use in Windows.
  ☆438Updated 2 months ago
• OpenBMB / MiniCPM-CookBook
  This is a user guide for the MiniCPM and MiniCPM-V series of small language models (SLMs) developed by ModelBest. “面壁小钢炮” focuses on achi…
  ☆223Updated 5 months ago
• ABexit / ASR-LLM-TTS
  This is a speech interaction system built on an open-source model, integrating ASR, LLM, and TTS in sequence. The ASR model is SenceVoice…
  ☆586Updated last month
• shell-nlp / gpt_server
  gpt_server是一个用于生产级部署LLMs或Embedding的开源框架。
  ☆163Updated this week
• CrazyBoyM / phi3-Chinese
  Phi3 中文后训练模型仓库
  ☆320Updated 4 months ago
• liuhuapiaoyuan / MinerU-webui
  MinerU是一款开源的高质量PDF解析工具，基于深度学习技术，可自动提取PDF文档中的文字、表格、图片、公式等内容，并提供丰富的分析、统计、搜索等功能。 本项目为其提供一个简化版本的WebUI，方便用户上传PDF文件，并实时展示提取结果。
  ☆161Updated 3 months ago
• pengzhendong / streaming-sensevoice
  Pseudo Streaming SenseVoice with Hotwords
  ☆233Updated 2 weeks ago
• brightwang / graphrag-dify
  ☆227Updated 3 months ago
• seanzhang-zhichen / llama3-chinese
  Llama3-Chinese是以Meta-Llama-3-8B为底座，使用 DORA + LORA+ 的训练方法，在50w高质量中文多轮SFT数据 + 10w英文多轮SFT数据 + 2000单轮自我认知数据训练而来的大模型。
  ☆294Updated 11 months ago
• HerSophia / LightRAG-for-OpenAI-Standard-Frontend
  支持OpenAI标准响应格式，可部署为服务并连接任意支持该格式的前端服务
  ☆30Updated 2 months ago
• NanGePlus / KagTest
  KAG开源框架介绍及使用KAG实现知识增强生成应用(产品模式测试、开发者模式测试)，KAG是OpenSPG发布v0.5版本中推出的知识增强生成（KAG）的专业领域知识服务框架，旨在充分利用知识图谱和向量检索的优势，增强大型语言模型和知识图谱，以解决 RAG 挑战
  ☆75Updated 2 months ago
• modelscope / evalscope
  A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
  ☆726Updated this week
• mozhang-ah / aiWatchdog
  ☆158Updated 2 weeks ago
• likelovewant / ollama-for-amd
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.by adding more amd gpu support.
  ☆956Updated this week
• wyf3 / llm_related
  记录大模型相关的一些知识和方法
  ☆1,144Updated last week
• infinigence / Infini-Megrez-Omni
  ☆218Updated last month