Alternatives and similar repositories for deepseek.cpp

Users that are interested in deepseek.cpp are comparing it to the libraries listed below

• andrewkchan / yalm
  Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O
  ☆550Updated 4 months ago
• yassa9 / qwen600
  Static suckless single batch CUDA-only qwen3-0.6B mini inference engine
  ☆543Updated 4 months ago
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆225Updated 3 weeks ago
• modal-labs / gpu-glossary
  GPU documentation for humans
  ☆514Updated last week
• Anemll / mlx-rdma
  experiments with MLX
  ☆66Updated last month
• horus-ai-labs / DistillFlow
  Library for model distillation
  ☆160Updated 5 months ago
• powerserve-project / PowerServe
  High-speed and easy-use LLM serving framework for local deployment
  ☆145Updated 5 months ago
• danbev / learning-ai
  Notes and exploration code for learning about AI/ML
  ☆207Updated last week
• ling0322 / libllm
  Efficient inference of large language models.
  ☆149Updated 4 months ago
• google / minja
  A minimalistic C++ Jinja templating engine for LLM chat templates
  ☆202Updated 4 months ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆785Updated this week
• likejazz / llama3.cuda
  llama3.cuda is a pure C/CUDA implementation for Llama 3 model.
  ☆350Updated 9 months ago
• microsoft / T-MAC
  Low-bit LLM inference on CPU/NPU with lookup table
  ☆916Updated 8 months ago
• Tencent / hpc-ops
  High Performance LLM Inference Operator Library
  ☆695Updated this week
• Chen-zexi / vllm-cli
  A command-line interface tool for serving LLM using vLLM.
  ☆468Updated last week
• intel / auto-round
  🎯An accuracy-first, highly efficient quantization toolkit for LLMs, designed to minimize quality degradation across Weight-Only Quantiza…
  ☆839Updated this week
• deepseek-ai / LPLB
  An early research stage expert-parallel load balancer for MoE models based on linear programming.
  ☆495Updated 2 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆390Updated 2 months ago
• adriancable / qwen3.c
  Local Qwen3 LLM inference. One easy-to-understand file of C source with no dependencies.
  ☆156Updated 7 months ago
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆295Updated 2 months ago
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆60Updated last year
• Infini-AI-Lab / UMbreLLa
  LLM Inference on consumer devices
  ☆129Updated 10 months ago
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆260Updated 11 months ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆674Updated 9 months ago
• NVlabs / vibetensor
  Our first fully AI generated deep learning system
  ☆481Updated this week
• micytao / vllm-playground
  A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes…
  ☆351Updated this week
• thad0ctor / llama-server-launcher
  ☆107Updated last week
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆104Updated last week
• NimbleEdge / sparse_transformers
  Sparse Inferencing for transformer based LLMs
  ☆218Updated 5 months ago
• xlite-dev / ffpa-attn
  🤖FFPA: Extend FlashAttention-2 with Split-D, ~O(1) SRAM complexity for large headdim, 1.8x~3x↑🎉 vs SDPA EA.
  ☆248Updated 2 weeks ago