Alternatives and similar repositories for optimize_llamacpp_ngl

Users that are interested in optimize_llamacpp_ngl are comparing it to the libraries listed below

• lucasavila00 / LmScript
  Controllable Language Model Interactions in TypeScript
  ☆10Updated last year
• silphendio / sliced_llama
  Simple LLM inference server
  ☆20Updated last year
• TLAMHutto / ollamaChat
  ☆23Updated last year
• josephrocca / rwkv-v4-web
  BlinkDL's RWKV-v4 running in the browser
  ☆48Updated 2 years ago
• ddh0 / easy-llama
  Python package wrapping llama.cpp for on-device LLM inference
  ☆100Updated 3 months ago
• xaedes / llama.cpp
  Port of Facebook's LLaMA model in C/C++
  ☆21Updated 2 years ago
• wassname / prob_jsonformer
  Generate Structured JSON with probs from Language Models
  ☆17Updated 10 months ago
• astramind-ai / Pulsar
  The hearth of The Pulsar App, fast, secure and shared inference with modern UI
  ☆59Updated last year
• CharlesMod / quantizeHFmodel
  Accepts a Hugging Face model URL, automatically downloads and quantizes it using Bits and Bytes.
  ☆38Updated last year
• fishiatee / yawullm
  Yet Another (LLM) Web UI, made with Gemini
  ☆12Updated last year
• attashe / ModifiedBeamSampler
  Modified Beam Search with periodical restart
  ☆12Updated last year
• bjj / exllamav2-openai-server
  An OpenAI API compatible LLM inference server based on ExLlamaV2.
  ☆25Updated 2 years ago
• tercumantanumut / GameCompanionAI
  Game Companion AI is an advanced application designed to enhance the gaming experience by providing real-time analysis and interpretation…
  ☆54Updated last year
• grctest / Electron-BitNet
  Running Microsoft's BitNet via Electron, React & Astro
  ☆52Updated 4 months ago
• jukofyork / transplant-vocab
  Transplants vocabulary between language models, enabling the creation of draft models for speculative decoding WITHOUT retraining.
  ☆49Updated 3 months ago
• stringandstickytape / MaxsAiStudio
  A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.
  ☆35Updated 3 months ago
• remichu-ai / gallamaUI
  ☆23Updated 2 months ago
• electroglyph / quant_clone
  Generate a llama-quantize command to copy the quantization parameters of any GGUF
  ☆30Updated 2 weeks ago
• tolitius / towel
  "a towel is about the most massively useful thing an interstellar AI hitchhiker can have"
  ☆48Updated last year
• Hellisotherpeople / llm_steer-oobabooga
  Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…
  ☆44Updated last year
• william-murray1204 / stable-diffusion-cpp-python
  stable-diffusion.cpp bindings for python
  ☆98Updated this week
• HabermannR / Fantasy-Tribe-Game
  LLM backed Fantasy Tribe Game
  ☆19Updated last year
• cp3249 / splaa
  SPLAA is an AI assistant framework that utilizes voice recognition, text-to-speech, and tool-calling capabilities to provide a conversati…
  ☆28Updated 9 months ago
• reka-ai / rekaquant
  ☆63Updated 7 months ago
• RobViren / kokovoicelab
  A cli app for experimenting with kokoro voice creating and mixing using the available voices to interpolate new ones
  ☆36Updated last year
• nath1295 / LLMFlex
  A python package for developing AI applications with local LLMs.
  ☆150Updated last year
• LuciAkirami / liva
  LIVA - Local Intelligent Voice Assistant
  ☆61Updated last year
• Uminosachi / open-llm-webui
  This repository contains a web application designed to execute relatively compact, locally-operated Large Language Models (LLMs).
  ☆43Updated last week
• ngxson / ggml-easy
  Thin wrapper around GGML to make life easier
  ☆42Updated 3 months ago
• xhedit / quantkit
  cli tool to quantize gguf, gptq, awq, hqq and exl2 models
  ☆78Updated last year