kvcache-ai / ktransformersLinks
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
☆16,264Updated this week
Alternatives and similar repositories for ktransformers
Users that are interested in ktransformers are comparing it to the libraries listed below
Sorting:
- FlashMLA: Efficient Multi-head Latent Attention Kernels☆11,931Updated 2 weeks ago
- Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-p…☆8,883Updated this week
- SGLang is a fast serving framework for large language models and vision language models.☆21,945Updated this week
- Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.☆12,764Updated 3 months ago
- GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型☆6,997Updated 5 months ago
- GPU cluster manager for optimized AI model deployment☆4,298Updated this week
- DeepEP: an efficient expert-parallel communication library☆8,826Updated last week
- Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation☆7,947Updated 7 months ago
- Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3, Qwen3-MoE, DeepSeek-R1, GLM4.5, InternLM3, Llama4, ...) and 300+ MLLMs (…☆11,815Updated this week
- LMDeploy is a toolkit for compressing, deploying, and serving LLMs.☆7,437Updated this week
- OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, …☆6,479Updated this week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆4,471Updated this week
- ☆4,576Updated last week
- The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.☆20,037Updated last month
- Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek-R1, Qwen3, Gemma 3, TTS 2x faster with 70% less VRAM.☆49,952Updated this week
- Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)☆64,621Updated this week
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model☆4,987Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆66,313Updated this week
- Qwen3 is the large language model series developed by Qwen team, Alibaba Cloud.☆25,863Updated 2 months ago
- Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.☆18,347Updated this week
- 分享一些好用的 Dify DSL 工作流程，自用、学习两相宜。 Sharing some Dify workflows.☆9,840Updated 6 months ago
- Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.☆7,262Updated last month
- Easily build AI systems with Evals, RAG, Agents, fine-tuning, synthetic data, and more.☆4,490Updated last week
- No fortress, purely open ground. OpenManus is Coming.☆51,425Updated last month
- 🍒 Cherry Studio is a desktop client that supports for multiple LLM providers.☆37,000Updated this week
- Question and Answer based on Anything.☆13,800Updated 9 months ago
- A Fundamental End-to-End Speech Recognition Toolkit and Open Source SOTA Pretrained Models, Supporting Speech Recognition, Voice Activity…☆14,176Updated this week
- Qwen2.5-Omni is an end-to-end multimodal model by Qwen team at Alibaba Cloud, capable of understanding text, audio, vision, video, and pe…☆3,859Updated 6 months ago
- Production-ready platform for agentic workflow development.☆122,824Updated this week
- A powerful tool for creating fine-tuning datasets for LLM☆12,547Updated last month