Alternatives and similar repositories for llm.cpp

Users that are interested in llm.cpp are comparing it to the libraries listed below

• salykova / sgemm.cu
  High-Performance FP32 GEMM on CUDA devices
  ☆117Updated last year
• GaoYusong / llm.cpp
  A C++ port of karpathy/llm.c features a tiny torch library while maintaining overall simplicity.
  ☆42Updated last year
• ARM-software / kleidiai
  This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai
  ☆113Updated last week
• triton-lang / kernels
  ☆104Updated last year
• andrewkchan / yalm
  Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O
  ☆550Updated 4 months ago
• vllm-project / tpu-inference
  TPU inference for vLLM, with unified JAX and PyTorch support.
  ☆228Updated this week
• perplexityai / pplx-kernels
  Perplexity GPU Kernels
  ☆554Updated 3 months ago
• unixpickle / learn-ptx
  Learning about CUDA by writing PTX code.
  ☆152Updated last year
• HabanaAI / SynapseAI_Core
  SynapseAI Core is a reference implementation of the SynapseAI API running on Habana Gaudi
  ☆42Updated last year
• microsoft / ark
  A GPU-driven system framework for scalable AI applications
  ☆124Updated last year
• gevtushenko / llm.c
  LLM training in simple, raw C/CUDA
  ☆112Updated last year
• Deep-Learning-Profiling-Tools / triton-viz
  ☆286Updated this week
• dropbox / gemlite
  Fast low-bit matmul kernels in Triton
  ☆427Updated last week
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆64Updated 7 months ago
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆260Updated last year
• InternLM / turbomind
  ☆96Updated 10 months ago
• gpu-mode / triton-index
  Cataloging released Triton kernels.
  ☆292Updated 4 months ago
• gpu-mode / ring-attention
  ring-attention experiments
  ☆165Updated last year
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆111Updated last week
• vedantroy / gpu_kernels
  ☆27Updated 2 years ago
• rogerallen / llama2.cu
  Inference Llama 2 in one file of pure C & one file with CUDA
  ☆32Updated 2 years ago
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆394Updated 7 months ago
• meta-pytorch / tritonbench
  Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.
  ☆324Updated this week
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆125Updated last year
• Dao-AILab / sonic-moe
  Accelerating MoE with IO and Tile-aware Optimizations
  ☆569Updated 2 weeks ago
• pranjalssh / fast.cu
  Fastest kernels written from scratch
  ☆532Updated 4 months ago
• cchan / tccl
  extensible collectives library in triton
  ☆95Updated 10 months ago
• tspeterkim / paged-attention-minimal
  a minimal cache manager for PagedAttention, on top of llama3.
  ☆135Updated last year
• usyd-fsalab / fp6_llm
  An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).
  ☆276Updated 6 months ago
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆55Updated 5 months ago