Alternatives and similar repositories for LLM-inference-speed-benchmarks:

Users that are interested in LLM-inference-speed-benchmarks are comparing it to the libraries listed below

• slashml / awesome-small-language-models
  ☆21Updated 5 months ago
• rodrigobaron / anthill
  ☆25Updated last week
• cwhy / rwkv-decon
  Trying to deconstruct RWKV in understandable terms
  ☆14Updated last year
• calmstate / Itinerant
  A sleek, customizable interface for managing LLMs with responsive design and easy agent personalization.
  ☆12Updated 5 months ago
• slashml / awesome-finetuning
  ☆27Updated 5 months ago
• stringandstickytape / MaxsAiStudio
  A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.
  ☆29Updated this week
• attashe / ModifiedBeamSampler
  Modified Beam Search with periodical restart
  ☆12Updated 4 months ago
• Toy-97 / Chat-WebUI
  Chat WebUI is an easy-to-use user interface for interacting with AI, and it comes with multiple useful built-in tools.
  ☆15Updated 2 months ago
• pepijndevos / llama_multiserver
  A proxy that hosts multiple single-model runners such as LLama.cpp and vLLM
  ☆12Updated last month
• sammcj / ollama-artefacts
  Build HTML artefacts with Ollama
  ☆11Updated last month
• agokrani / distillKitPlus
  Easy to use, High Performant Knowledge Distillation for LLMs
  ☆40Updated 3 weeks ago
• FishiaT / yawullm
  Yet Another (LLM) Web UI, made with Gemini
  ☆11Updated last month
• Zyphra / zcookbook
  Training hybrid models for dummies.
  ☆18Updated 2 weeks ago
• tonymantoan / local_graphiti
  ☆15Updated last month
• mesolitica / transformers-openai-api
  Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
  ☆19Updated 2 months ago
• kyegomez / OpenStrawberry
  An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO
  ☆27Updated this week
• astramind-ai / Pulsar
  The hearth of The Pulsar App, fast, secure and shared inference with modern UI
  ☆53Updated last month
• bdytx5 / open_answer_engine
  ☆21Updated 5 months ago
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆12Updated 5 months ago
• LAION-AI / Desktop-BUD-E_V1.0
  BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the cre…
  ☆17Updated 3 months ago
• Edw590 / VISOR---A-Voice-Assistant
  V.I.S.O.R., my in-development AI-powered voice assistant with integrated memory!
  ☆32Updated last month
• microsoft / Structured-Entity-Extraction
  Code for the EMNLP'24 paper "Learning to Extract Structured Entities Using Language Models"
  ☆24Updated 2 weeks ago
• arthurwolf / llmi
  Large-Language-Model to Machine Interface project.
  ☆17Updated last year
• lechmazur / writing
  This benchmark tests how well LLMs incorporate a set of 10 mandatory story elements (characters, objects, core concepts, attributes, moti…
  ☆35Updated last week
• shinomakoi / AI-Messenger
  A QT GUI for large language models
  ☆28Updated last year
• jllllll / GPTQ-for-LLaMa-CUDA
  A combination of Oobabooga's fork and the main cuda branch of GPTQ-for-LLaMa in a package format.
  ☆22Updated last year
• thisishugow / create-llama-ollama
  a simple create-llama template using llama-index v0.10 and integrated with Ollama
  ☆10Updated 8 months ago
• lechmazur / nyt-connections
  Benchmark that evaluates LLMs using 436 NYT Connections puzzles
  ☆12Updated this week
• JakeFurtaw / Chat-RAG
  Uses a Gradio interface to stream coding related responses from local and cloud based large language models. Pulls context from GitHub Re…
  ☆19Updated 4 months ago