Alternatives and similar repositories for edge-slm:

Users that are interested in edge-slm are comparing it to the libraries listed below

• deepsense-ai / db-ally
  Efficient, consistent and secure library for querying structured data with natural language
  ☆153Updated last week
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆277Updated this week
• dusty-nv / NanoLLM
  Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector…
  ☆255Updated 6 months ago
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆168Updated 3 weeks ago
• PINTO0309 / whisper-onnx-tensorrt
  ONNX and TensorRT implementation of Whisper
  ☆61Updated last year
• triton-inference-server / vllm_backend
  ☆246Updated last week
• saic-fi / MobileQuant
  [EMNLP Findings 2024] MobileQuant: Mobile-friendly Quantization for On-device Language Models
  ☆56Updated 7 months ago
• Vaibhavs10 / optimise-my-whisper
  ☆204Updated 11 months ago
• deepsense-ai / ds-template
  Template for professional data science and python applications made by deepsense.ai
  ☆31Updated last week
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆214Updated 4 months ago
• moxin-org / Moxin-LLM
  Moxin is a family of fully open-source and reproducible LLMs
  ☆87Updated this week
• GaiZhenbiao / Phi3V-Finetuning
  Parameter-efficient finetuning script for Phi-3-vision, the strong multimodal language model by Microsoft.
  ☆58Updated 10 months ago
• l4b4r4b4b4 / AIDocks
  LLM-Training-API: Including Embeddings & ReRankers, mergekit, LaserRMT
  ☆27Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆106Updated last week
• center-for-humans-and-machines / transformer-heads
  Toolkit for attaching, training, saving and loading of new heads for transformer models
  ☆275Updated last month
• Entropy-xcy / bitnet158
  ☆69Updated last year
• stevelaskaridis / awesome-mobile-llm
  Awesome Mobile LLMs
  ☆169Updated last month
• NVIDIA-AI-IOT / whisper_trt
  A project that optimizes Whisper for low latency inference using NVIDIA TensorRT
  ☆81Updated 6 months ago
• RobinQu / instinct.cpp
  instinct.cpp provides ready to use alternatives to OpenAI Assistant API and built-in utilities for developing AI Agent applications (RAG,…
  ☆46Updated 9 months ago
• hkproj / mistral-src-commented
  Reference implementation of Mistral AI 7B v0.1 model.
  ☆28Updated last year
• nyunAI / PruneGPT
  ☆53Updated 10 months ago
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆295Updated this week
• AlexBodner / How_Much_VRAM
  ☆100Updated 7 months ago
• cindysridykhan / instruct_storyteller_tinyllama2
  Training and Fine-tuning an llm in Python and PyTorch.
  ☆41Updated last year
• NVIDIA / trt-llm-as-openai-windows
  This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows inste…
  ☆120Updated last year
• AIWintermuteAI / whispercpp
  Pybind11 bindings for Whisper.cpp
  ☆55Updated 3 weeks ago
• PINTO0309 / whisper-onnx-cpu
  ONNX implementation of Whisper. PyTorch free.
  ☆94Updated 5 months ago
• speechmatics / ctranslate2_triton_backend
  Triton backend for https://github.com/OpenNMT/CTranslate2
  ☆35Updated last year
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆202Updated last year
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆136Updated 9 months ago