Alternatives and similar repositories for llm.cpp

Users that are interested in llm.cpp are comparing it to the libraries listed below

• GaoYusong / llm.cpp
  A C++ port of karpathy/llm.c features a tiny torch library while maintaining overall simplicity.
  ☆42Updated last year
• salykova / sgemm.cu
  High-Performance FP32 GEMM on CUDA devices
  ☆117Updated last year
• andrewkchan / yalm
  Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O
  ☆550Updated 4 months ago
• rogerallen / llama2.cu
  Inference Llama 2 in one file of pure C & one file with CUDA
  ☆32Updated 2 years ago
• amd / ZenDNN
  ☆137Updated last week
• microsoft / ark
  A GPU-driven system framework for scalable AI applications
  ☆124Updated last year
• MetaX-MACA / FlashMLA
  Fast and efficient attention method exploration and implementation.
  ☆25Updated 10 months ago
• gevtushenko / llm.c
  LLM training in simple, raw C/CUDA
  ☆112Updated last year
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆111Updated last week
• intel / intel-xpu-backend-for-triton
  OpenAI Triton backend for Intel® GPUs
  ☆226Updated last week
• meta-pytorch / triton-cpu
  An experimental CPU backend for Triton (https//github.com/openai/triton)
  ☆49Updated 5 months ago
• SzymonOzog / GPU_Programming
  ☆89Updated 2 months ago
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆64Updated 7 months ago
• tspeterkim / paged-attention-minimal
  a minimal cache manager for PagedAttention, on top of llama3.
  ☆135Updated last year
• kroggen / mamba.c
  Inference of Mamba and Mamba2 models in pure C
  ☆196Updated 2 weeks ago
• sgl-project / sgl-learning-materials
  Materials for learning SGLang
  ☆738Updated last month
• gpu-mode / triton-index
  Cataloging released Triton kernels.
  ☆292Updated 5 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆125Updated last year
• unixpickle / learn-ptx
  Learning about CUDA by writing PTX code.
  ☆152Updated last year
• perplexityai / pplx-kernels
  Perplexity GPU Kernels
  ☆554Updated 3 months ago
• meta-pytorch / torchcomms
  torchcomms: a modern PyTorch communications API
  ☆327Updated this week
• Shenggan / awesome-distributed-ml
  A curated list of awesome projects and papers for distributed training or inference
  ☆265Updated last year
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆204Updated last week
• justinchuby / onnx-safetensors
  Use safetensors with ONNX 🤗
  ☆84Updated 3 weeks ago
• NVIDIA / nvshmem
  NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…
  ☆462Updated last month
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆457Updated 8 months ago
• vllm-project / tpu-inference
  TPU inference for vLLM, with unified JAX and PyTorch support.
  ☆228Updated this week
• tgautam03 / xGeMM
  Accelerated General (FP32) Matrix Multiplication from scratch in CUDA
  ☆182Updated last year
• dropbox / gemlite
  Fast low-bit matmul kernels in Triton
  ☆427Updated last week
• staghado / vit.cpp
  Inference Vision Transformer (ViT) in plain C/C++ with ggml
  ☆306Updated last year