Alternatives and similar repositories for ggml:

Users that are interested in ggml are comparing it to the libraries listed below

• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆88Updated last week
• deepinfra / deepctl
  Command line tool for Deep Infra cloud ML inference service
  ☆26Updated 7 months ago
• nomic-ai / kompute
  General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). …
  ☆43Updated 3 months ago
• the-crypt-keeper / ggml-downloader
  Simple, Fast, Parallel Huggingface GGML model downloader written in python
  ☆24Updated last year
• ajtejankar / mixtral-vis-moe
  Visualize expert firing frequencies across sentences in the Mixtral MoE model
  ☆17Updated last year
• mistralai / vllm-release
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆51Updated last year
• hscspring / llama.np
  Inference Llama/Llama2/Llama3 Modes in NumPy
  ☆20Updated last year
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆52Updated 9 months ago
• intellectronica / battle-of-the-semantics
  GraphRag vs Embeddings
  ☆13Updated 6 months ago
• teknium1 / LLM-Logbook
  Public reports detailing responses to sets of prompts by Large Language Models.
  ☆28Updated 2 weeks ago
• KerfuffleV2 / gguf-tools
  Some random tools for working with the GGUF file format
  ☆24Updated last year
• laelhalawani / gguf_modeldb
  A quick and optimized solution to manage llama based gguf quantized models, download gguf files, retreive messege formatting, add more mo…
  ☆12Updated last year
• samuel-vitorino / lm.rs-webui
  Light WebUI for lm.rs
  ☆23Updated 3 months ago
• NolanoOrg / SpectraSuite
  ☆44Updated 6 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆74Updated 4 months ago
• facebookresearch / dual-system-for-visual-language-reasoning
  Github repo for Peifeng's internship project
  ☆13Updated last year
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆12Updated 5 months ago
• moxin-org / Moxin-LLM
  ☆82Updated last month
• nova-land / gbnf-compiler
  Plug n Play GBNF Compiler for llama.cpp
  ☆23Updated last year
• crashr / gppm
  GPU Power and Performance Manager
  ☆51Updated 3 months ago
• RobinQu / instinct.cpp
  instinct.cpp provides ready to use alternatives to OpenAI Assistant API and built-in utilities for developing AI Agent applications (RAG,…
  ☆42Updated 6 months ago
• groq / groqflow
  GroqFlow provides an automated tool flow for compiling machine learning and linear algebra workloads into Groq programs and executing tho…
  ☆106Updated last month
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆87Updated this week
• gau-nernst / quantized-training
  Explore training for quantized models
  ☆12Updated 2 weeks ago
• abhisheknair10 / Llama3.cu
  Lightweight Llama 3 8B Inference Engine in CUDA C
  ☆42Updated 2 weeks ago
• cwhy / rwkv-decon
  Trying to deconstruct RWKV in understandable terms
  ☆14Updated last year
• mscheong01 / speculative_decoding.c
  minimal C implementation of speculative decoding based on llama2.c
  ☆18Updated 6 months ago
• xaedes / llama.cpp
  Port of Facebook's LLaMA model in C/C++
  ☆20Updated last year
• rafacelente / bllama
  1.58-bit LLaMa model
  ☆80Updated 9 months ago