dusty-nv / NanoLLM
Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
☆239Updated 4 months ago
Alternatives and similar repositories for NanoLLM:
Users that are interested in NanoLLM are comparing it to the libraries listed below
- ☆98Updated this week
- A tutorial introducing knowledge distillation as an optimization technique for deployment on NVIDIA Jetson☆171Updated last year
- A utility library to help integrate Python applications with Metropolis Microservices for Jetson☆12Updated 2 months ago
- A reference application for a local AI assistant with LLM and RAG☆103Updated 2 months ago
- ☆92Updated 5 months ago
- Zero-copy multimodal vector DB with CUDA and CLIP/SigLIP☆46Updated 8 months ago
- A collection of reference AI microservices and workflows for Jetson Platform Services☆34Updated 3 weeks ago
- This repo has the code of the 3 demos I presented at Google Gemma2 DevDay Tokyo, using Gemma2 on a Jetson Orin Nano device.☆35Updated 3 months ago
- A reference example for integrating NanoOwl with Metropolis Microservices for Jetson☆30Updated 8 months ago
- ASR/NLP/TTS deep learning inference library for NVIDIA Jetson using PyTorch and TensorRT☆199Updated last year
- A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.☆300Updated 2 weeks ago
- Quick start scripts and tutorial notebooks to get started with TAO Toolkit☆66Updated 5 months ago
- Creation of annotated datasets from scratch using Generative AI and Foundation Computer Vision models☆111Updated this week
- Using FastChat-T5 Large Language Model, Vosk API for automatic speech recognition, and Piper for text-to-speech☆117Updated last year
- A repository for affordable, easy-to-assemble robot arms designed for teleoperation applications.☆73Updated last week
- The jetson-examples repository by Seeed Studio offers a seamless, one-line command deployment to run vision AI and Generative AI models o…☆145Updated 2 weeks ago
- Beginner's Guide to reComputer Jetson☆105Updated 4 months ago
- A version 1.1 of the Alexander Koch low cost robot arm with some small changes.☆544Updated 5 months ago
- The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment☆381Updated 2 weeks ago
- ☆169Updated last year
- Embed arbitrary modalities (images, audio, documents, etc) into large language models.☆177Updated 10 months ago
- Collection of reference workflows for building intelligent agents with NIMs☆146Updated last month
- Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inferen…☆54Updated this week
- From scratch implementation of a vision language model in pure PyTorch☆194Updated 9 months ago
- A project demonstrating how to make DeepStream docker images.☆68Updated 2 months ago
- A tool to configure, launch and manage your machine learning experiments.☆117Updated this week
- Which model is the best at object detection? Which is best for small or large objects? We compare the results in a handy leaderboard.☆63Updated this week
- Sample app code for deploying TAO Toolkit trained models to Triton☆86Updated 5 months ago
- TAO Toolkit deep learning networks with PyTorch backend☆91Updated 3 months ago
- OpenAI compatible API for TensorRT LLM triton backend☆191Updated 6 months ago