Alternatives and similar repositories for tensorlink

Users that are interested in tensorlink are comparing it to the libraries listed below

• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆236Updated this week
• vtuber-plan / olah
  Self-hosted huggingface mirror service. 自建huggingface镜像服务。
  ☆176Updated 2 months ago
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆270Updated last year
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆185Updated last week
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆134Updated last week
• leptonai / gpud
  GPUd automates monitoring, diagnostics, and issue identification for GPUs
  ☆387Updated this week
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆209Updated 11 months ago
• 01-ai / Descartes
  ☆110Updated last year
• ninehills / llm-inference-benchmark
  LLM Inference benchmark
  ☆422Updated 11 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆219Updated 7 months ago
• MoonshotAI / moonpalace
  MoonPalace（月宫）是由 Moonshot AI 月之暗面提供的 API 调试工具。
  ☆197Updated 6 months ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆243Updated last year
• tenclass / clink
  Implementation of remote CUDA/OpenCL protocol
  ☆36Updated last month
• OpenCSGs / llm-inference
  llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deploy…
  ☆85Updated last year
• mayooot / gpu-docker-api
  Easier than K8s to lift and lower the gpu number of docker container and scale capacity size of volume.
  ☆77Updated last year
• matpool / mpu
  A shim driver allows in-docker nvidia-smi showing correct process list without modify anything
  ☆88Updated 2 weeks ago
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆242Updated 4 months ago
• RealAlexandreAI / json-repair
  🔧 Repair JSON！Solution for JSON Anomalies from LLMs.
  ☆268Updated last month
• horus-ai-labs / DistillFlow
  Library for model distillation
  ☆146Updated 5 months ago
• star-whale / starwhale
  an MLOps/LLMOps platform
  ☆230Updated 6 months ago
• Yard1 / Ray-DeepSpeed-Inference
  ☆17Updated 2 years ago
• chenyangMl / llama2.c-zh
  支持中文场景的的小语言模型 llama2.c-zh
  ☆147Updated last year
• InftyAI / llmaz
  ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
  ☆224Updated last week
• kubeagi / arcadia
  A diverse, simple, and secure all-in-one LLMOps platform
  ☆107Updated 9 months ago
• pokerfaceSad / GPUMounter
  A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod
  ☆126Updated 3 years ago
• intel / xFasterTransformer
  ☆428Updated last week
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆160Updated last week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆129Updated last week
• vectorch-ai / ScaleLLM
  A high-performance inference system for large language models, designed for production environments.
  ☆455Updated this week
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆94Updated this week