kvcache-ai / ktransformersLinks
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
☆16,129Updated this week
Alternatives and similar repositories for ktransformers
Users that are interested in ktransformers are comparing it to the libraries listed below
Sorting:
- GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型☆6,947Updated 5 months ago
- Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-p…☆8,816Updated this week
- SGLang is a fast serving framework for large language models and vision language models.☆20,874Updated this week
- GPU cluster manager for optimized AI model deployment☆4,147Updated this week
- Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 500+ LLMs (Qwen3, Qwen3-MoE, Llama4, GLM4.5, InternLM3, DeepSeek-R1, ...) and 200+ MLLMs (…☆11,418Updated this week
- Qwen3 is the large language model series developed by Qwen team, Alibaba Cloud.☆25,625Updated last month
- FlashMLA: Efficient Multi-head Latent Attention Kernels☆11,896Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆64,758Updated this week
- Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek-R1, Qwen3, Gemma 3, TTS 2x faster with 70% less VRAM.☆49,033Updated this week
- LMDeploy is a toolkit for compressing, deploying, and serving LLMs.☆7,357Updated this week
- DeepEP: an efficient expert-parallel communication library☆8,788Updated this week
- Easily build AI systems with Evals, RAG, Agents, fine-tuning, synthetic data, and more.☆4,444Updated last week
- Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)☆63,706Updated this week
- verl: Volcano Engine Reinforcement Learning for LLMs☆17,204Updated this week
- Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.☆12,558Updated 2 months ago
- DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling☆5,944Updated this week
- A powerful tool for creating fine-tuning datasets for LLM☆12,279Updated last week
- Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.☆16,985Updated last week
- Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.☆7,222Updated 2 weeks ago
- Retrieval and Retrieval-augmented LLMs☆10,959Updated last month
- OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, …☆6,391Updated this week
- Fully open reproduction of DeepSeek-R1☆25,717Updated 2 weeks ago
- MiniCPM-V 4.5: A GPT-4o Level MLLM for Single Image, Multi Image and High-FPS Video Understanding on Your Phone☆22,330Updated 2 months ago
- The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.☆19,893Updated last week
- Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation☆7,937Updated 6 months ago
- ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型☆13,734Updated 10 months ago
- An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & vLLM & Ray & Dynamic Sampling & Asy…☆8,535Updated last month
- Toolkit for linearizing PDFs for LLM datasets/training☆16,165Updated this week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆4,357Updated this week
- 🚀 「大模型」1小时从0训练26M参数的视觉多模态VLM！🌏 Train a 26M-parameter VLM from scratch in just 1 hours!☆5,588Updated this week