Alternatives and similar repositories for ggml

Users that are interested in ggml are comparing it to the libraries listed below

• nomic-ai / kompute
  General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). …
  ☆51Updated 11 months ago
• antirez / LLM-FTC-sampling
  First token cutoff sampling inference example
  ☆30Updated 2 years ago
• antirez / gguf-tools
  GGUF implementation in C as a library and a tools CLI program
  ☆303Updated 5 months ago
• abetlen / ggml-python
  Python bindings for ggml
  ☆147Updated last year
• xyzhang626 / embeddings.cpp
  ggml implementation of embedding models including SentenceTransformer and BGE
  ☆63Updated 2 years ago
• akx / ggify
  Tool to download models from Huggingface Hub and convert them to GGML/GGUF for llama.cpp
  ☆170Updated 9 months ago
• groq / groqflow
  GroqFlow provides an automated tool flow for compiling machine learning and linear algebra workloads into Groq programs and executing tho…
  ☆115Updated 6 months ago
• fblgit / tree-of-knowledge
  ToK aka Tree of Knowledge for Large Language Models LLM. It's a novel dataset that inspires knowledge symbolic correlation in simple inpu…
  ☆55Updated 2 years ago
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆93Updated this week
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆55Updated last year
• kroggen / mamba.c
  Inference of Mamba and Mamba2 models in pure C
  ☆196Updated 3 weeks ago
• vikhyat / mixtral-inference
  inference code for mixtral-8x7b-32kseqlen
  ☆105Updated 2 years ago
• jart / llama.cpp
  Port of Facebook's LLaMA model in C/C++
  ☆23Updated last year
• wozeparrot / tinyrwkv
  tinygrad port of the RWKV large language model.
  ☆45Updated 11 months ago
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆352Updated last year
• leloykun / llama2.cpp
  Inference Llama 2 in one file of pure C++
  ☆87Updated 2 years ago
• Artefact2 / llm-eval
  A super simple web interface to perform blind tests on LLM outputs.
  ☆29Updated last year
• pranavjad / tinyllama-bitnet
  Train your own small bitnet model
  ☆77Updated last year
• NolanoOrg / llama-int4-quant
  ☆26Updated 2 years ago
• janhq / cortex.llamacpp
  cortex.llamacpp is a high-efficiency C++ inference engine for edge computing. It is a dynamic library that can be loaded by any server a…
  ☆42Updated 7 months ago
• BlinkDL / nanoRWKV
  RWKV in nanoGPT style
  ☆197Updated last year
• argilla-io / notus
  Notus is a collection of fine-tuned LLMs using SFT, DPO, SFT+DPO, and/or any other RLHF techniques, while always keeping a data-first app…
  ☆169Updated 2 years ago
• mistralai / vllm-release
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆53Updated 2 years ago
• mzbac / mlx-moe
  Scripts to create your own moe models using mlx
  ☆90Updated last year
• fixie-ai / ai-benchmarks
  Benchmarking suite for popular AI APIs
  ☆88Updated last year
• petals-infra / health.petals.dev
  🏥 Health monitor for a Petals swarm
  ☆40Updated last year
• monatis / lmm.cpp
  Inference of Large Multimodal Models in C/C++. LLaVA and others
  ☆48Updated 2 years ago
• evintunador / minGemma
  a simplified version of Google's Gemma model to be used for learning
  ☆26Updated last year
• amd / ryzen-ai-documentation
  Onboarding documentation source for the AMD Ryzen™ AI Software Platform. The AMD Ryzen™ AI Software Platform enables developers to take…
  ☆92Updated last week
• 99991 / pygguf
  GGUF parser in Python
  ☆28Updated last year