dusty-nv / NanoLLMLinks
Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
☆309Updated 10 months ago
Alternatives and similar repositories for NanoLLM
Users that are interested in NanoLLM are comparing it to the libraries listed below
Sorting:
- ☆113Updated this week
- A tutorial introducing knowledge distillation as an optimization technique for deployment on NVIDIA Jetson☆208Updated last year
- A reference application for a local AI assistant with LLM and RAG☆114Updated 8 months ago
- ☆99Updated 11 months ago
- Quick start scripts and tutorial notebooks to get started with TAO Toolkit☆102Updated 3 weeks ago
- A collection of reference AI microservices and workflows for Jetson Platform Services☆48Updated 6 months ago
- Collection of reference workflows for building intelligent agents with NIMs☆169Updated 7 months ago
- A project that optimizes Whisper for low latency inference using NVIDIA TensorRT☆89Updated 10 months ago
- A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.☆362Updated 6 months ago
- Creation of annotated datasets from scratch using Generative AI and Foundation Computer Vision models☆124Updated last month
- This repo has the code of the 3 demos I presented at Google Gemma2 DevDay Tokyo, using Gemma2 on a Jetson Orin Nano device.☆58Updated last month
- A utility library to help integrate Python applications with Metropolis Microservices for Jetson☆15Updated 8 months ago
- Zero-copy multimodal vector DB with CUDA and CLIP/SigLIP☆61Updated 3 months ago
- Blueprint for Ingesting massive volumes of live or archived videos and extract insights for summarization and interactive Q&A☆203Updated last week
- High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices☆150Updated last month
- A project demonstrating how to make DeepStream docker images.☆82Updated 8 months ago
- TAO Toolkit deep learning networks with PyTorch backend☆100Updated 2 weeks ago
- A reference example for integrating NanoOwl with Metropolis Microservices for Jetson☆30Updated last year
- ASR/NLP/TTS deep learning inference library for NVIDIA Jetson using PyTorch and TensorRT☆217Updated last year
- Which model is the best at object detection? Which is best for small or large objects? We compare the results in a handy leaderboard.☆87Updated this week
- Beginner's Guide to reComputer Jetson☆111Updated 2 weeks ago
- ☆19Updated 4 months ago
- From scratch implementation of a vision language model in pure PyTorch☆235Updated last year
- A Toolkit to Help Optimize Onnx Model☆198Updated this week
- TinyChatEngine: On-Device LLM Inference Library☆887Updated last year
- An innovative library for efficient LLM inference via low-bit quantization☆349Updated 11 months ago
- A distilled Segment Anything (SAM) model capable of running real-time with NVIDIA TensorRT☆789Updated last year
- Take your LLM to the optometrist.☆36Updated 2 weeks ago
- An integration of Segment Anything Model, Molmo, and, Whisper to segment objects using voice and natural language.☆29Updated 6 months ago
- This repository provides optical character detection and recognition solution optimized on Nvidia devices.☆76Updated 3 months ago