dusty-nv / NanoLLMLinks
Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
☆344Updated last year
Alternatives and similar repositories for NanoLLM
Users that are interested in NanoLLM are comparing it to the libraries listed below
Sorting:
- GitHub repo for Jetson AI Lab☆129Updated this week
- Quick start scripts and tutorial notebooks to get started with TAO Toolkit☆129Updated last week
- A reference application for a local AI assistant with LLM and RAG☆117Updated last year
- ☆107Updated 2 months ago
- Collection of reference workflows for building intelligent agents with NIMs☆184Updated last year
- A collection of reference AI microservices and workflows for Jetson Platform Services☆51Updated 11 months ago
- This repo has the code of the 3 demos I presented at Google Gemma2 DevDay Tokyo, using Gemma2 on a Jetson Orin Nano device.☆60Updated 5 months ago
- A project that optimizes Whisper for low latency inference using NVIDIA TensorRT☆97Updated last year
- A utility library to help integrate Python applications with Metropolis Microservices for Jetson☆15Updated last year
- A project demonstrating how to make DeepStream docker images.☆92Updated 3 months ago
- High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices☆174Updated last week
- A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.☆395Updated 11 months ago
- Zero-copy multimodal vector DB with CUDA and CLIP/SigLIP☆64Updated 8 months ago
- Creation of annotated datasets from scratch using Generative AI and Foundation Computer Vision models☆132Updated last month
- Blueprint for Ingesting massive volumes of live or archived videos and extract insights for summarization and interactive Q&A☆373Updated last month
- ASR/NLP/TTS deep learning inference library for NVIDIA Jetson using PyTorch and TensorRT☆219Updated last year
- TAO Toolkit deep learning networks with PyTorch backend☆107Updated last month
- Simple and unified interface to zero-shot computer vision models curated for robotics use cases.☆167Updated 3 months ago
- A reference example for integrating NanoOwl with Metropolis Microservices for Jetson☆30Updated last year
- An open source light-weight and high performance inference framework for Hailo devices☆157Updated this week
- A Toolkit to Help Optimize Onnx Model☆308Updated this week
- ☆20Updated 9 months ago
- The jetson-examples repository by Seeed Studio offers a seamless, one-line command deployment to run vision AI and Generative AI models o…☆238Updated last month
- Using FastChat-T5 Large Language Model, Vosk API for automatic speech recognition, and Piper for text-to-speech☆128Updated 2 years ago
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆305Updated last year
- ☆33Updated last month
- The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…☆181Updated last month
- A tutorial for getting started with the Deep Learning Accelerator (DLA) on NVIDIA Jetson☆362Updated 3 years ago
- TinyChatEngine: On-Device LLM Inference Library☆939Updated last year
- Pre-built components and code samples to help you build and deploy production-grade AI applications with the OpenVINO™ Toolkit from Intel☆196Updated this week