Alternatives and similar repositories for edge-slm

Users that are interested in edge-slm are comparing it to the libraries listed below

• deepsense-ai / db-ally
  Efficient, consistent and secure library for querying structured data with natural language
  ☆159Updated last month
• Vaibhavs10 / optimise-my-whisper
  ☆204Updated last year
• stephenleo / llm-structured-output-benchmarks
  Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…
  ☆172Updated 8 months ago
• cognitivecomputations / spectrum
  ☆121Updated 2 months ago
• plaggy / fast-whisper-server
  ASR + diarization model server with speculative decoding
  ☆60Updated last year
• luweigen / whisper_streaming
  Whisper realtime streaming for long speech-to-text transcription and translation
  ☆117Updated last year
• docling-project / docling-parse
  Simple package to extract text with coordinates from programmatic PDFs
  ☆128Updated this week
• puppetm4st3r / baai_m3_simple_server
  This code sets up a simple yet robust server using FastAPI for handling asynchronous requests for embedding generation and reranking task…
  ☆69Updated last year
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆263Updated 7 months ago
• illuin-tech / vidore-benchmark
  Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.
  ☆208Updated this week
• lumpenspace / raft
  RAFT, or Retrieval-Augmented Fine-Tuning, is a method comprising of a fine-tuning and a RAG-based retrieval phase. It is particularly sui…
  ☆119Updated 9 months ago
• deepsense-ai / ragbits
  Building blocks for rapid development of GenAI applications
  ☆70Updated this week
• NVIDIA / logits-processor-zoo
  A collection of LogitsProcessors to customize and enhance LLM behavior for specific tasks.
  ☆288Updated this week
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆348Updated 9 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆217Updated 5 months ago
• ritabratamaiti / AnyModal
  AnyModal is a Flexible Multimodal Language Model Framework for PyTorch
  ☆95Updated 5 months ago
• pranavjad / tinyllama-bitnet
  Train your own small bitnet model
  ☆71Updated 7 months ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆86Updated 2 weeks ago
• AlexBodner / How_Much_VRAM
  ☆101Updated 9 months ago
• tiiuae / onebitllms
  Lightweight toolkit package to train and fine-tune 1.58bit Language models
  ☆69Updated 2 weeks ago
• unslothai / unsloth-zoo
  Utils for Unsloth
  ☆92Updated this week
• substratusai / stapi
  Sentence Transformers API: An OpenAI compatible embedding API server
  ☆59Updated 9 months ago
• ArturTanona / grpo_unsloth_docker
  ☆57Updated 3 months ago
• astramind-ai / BitMat
  An efficent implementation of the method proposed in "The Era of 1-bit LLMs"
  ☆154Updated 7 months ago
• khaimt / qa_expert
  This repo is for handling Question Answering, especially for Multi-hop Question Answering
  ☆67Updated last year
• PINTO0309 / whisper-onnx-cpu
  ONNX implementation of Whisper. PyTorch free.
  ☆97Updated 6 months ago
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆317Updated this week
• NVIDIA / trt-llm-as-openai-windows
  This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows inste…
  ☆121Updated last year
• skrbnv / javad
  ☆54Updated 4 months ago
• fairydreaming / llama.cpp
  LLM inference in C/C++
  ☆21Updated 2 months ago