dusty-nv / NanoLLMLinks
Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
☆271Updated 7 months ago
Alternatives and similar repositories for NanoLLM
Users that are interested in NanoLLM are comparing it to the libraries listed below
Sorting:
- ☆110Updated 2 months ago
- A tutorial introducing knowledge distillation as an optimization technique for deployment on NVIDIA Jetson☆196Updated last year
- A reference application for a local AI assistant with LLM and RAG☆112Updated 6 months ago
- A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.☆337Updated 4 months ago
- Quick start scripts and tutorial notebooks to get started with TAO Toolkit☆85Updated 9 months ago
- ☆96Updated 8 months ago
- This repo has the code of the 3 demos I presented at Google Gemma2 DevDay Tokyo, using Gemma2 on a Jetson Orin Nano device.☆45Updated last month
- Blueprint for Ingesting massive volumes of live or archived videos and extract insights for summarization and interactive Q&A☆97Updated last month
- A reference example for integrating NanoOwl with Metropolis Microservices for Jetson☆30Updated 11 months ago
- A utility library to help integrate Python applications with Metropolis Microservices for Jetson☆13Updated 5 months ago
- A collection of reference AI microservices and workflows for Jetson Platform Services☆39Updated 4 months ago
- ASR/NLP/TTS deep learning inference library for NVIDIA Jetson using PyTorch and TensorRT☆208Updated last year
- TAO Toolkit deep learning networks with PyTorch backend☆95Updated 6 months ago
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Tra…☆483Updated this week
- From scratch implementation of a vision language model in pure PyTorch☆220Updated last year
- Creation of annotated datasets from scratch using Generative AI and Foundation Computer Vision models☆118Updated 3 weeks ago
- Using FastChat-T5 Large Language Model, Vosk API for automatic speech recognition, and Piper for text-to-speech☆119Updated last year
- A tutorial for getting started with the Deep Learning Accelerator (DLA) on NVIDIA Jetson☆331Updated 3 years ago
- Zero-copy multimodal vector DB with CUDA and CLIP/SigLIP☆56Updated 3 weeks ago
- A family of compressed models obtained via pruning and knowledge distillation☆342Updated 6 months ago
- High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices☆139Updated 2 months ago
- Awesome Mobile LLMs☆195Updated 2 months ago
- A distilled Segment Anything (SAM) model capable of running real-time with NVIDIA TensorRT☆754Updated last year
- ☆173Updated last year
- NVIDIA DLA-SW, the recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.☆199Updated 11 months ago
- Beginner's Guide to reComputer Jetson☆107Updated 2 months ago
- A project demonstrating how to make DeepStream docker images.☆76Updated 6 months ago
- Quick exploration into fine tuning florence 2☆316Updated 8 months ago
- PyTorch installation wheels for Jetson Nano☆110Updated last year
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆285Updated last year