BestAnHongjun / LMDeploy-Jetson
Deploying LLMs offline on the NVIDIA Jetson platform marks the dawn of a new era in embodied intelligence, where devices can function independently without continuous internet access.
☆89Updated last year
Alternatives and similar repositories for LMDeploy-Jetson:
Users that are interested in LMDeploy-Jetson are comparing it to the libraries listed below
- 基于InternLM2大模型的离线具身智能导盲犬☆91Updated last year
- An onnx-based quantitation tool.☆71Updated last year
- ☆21Updated last year
- ☆132Updated last year
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆46Updated last year
- ☆39Updated 4 months ago
- ☆46Updated this week
- 基于昇腾310芯片的大语言模型部署☆18Updated 9 months ago
- ☆60Updated 8 months ago
- mllm-npu: training multimodal large language models on Ascend NPUs☆91Updated 7 months ago
- Inference code for LLaMA models☆118Updated last year
- 训练一个对中文支持更好的LLaVA模型，并开源训练代码和数据。☆53Updated 6 months ago
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆80Updated 2 months ago
- A light llama-like llm inference framework based on the triton kernel.☆100Updated 2 weeks ago
- [EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a V…☆436Updated 2 weeks ago
- Official PyTorch implementation of FlatQuant: Flatness Matters for LLM Quantization☆112Updated this week
- DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）☆151Updated last year
- Llama3 Streaming Chat Sample☆22Updated 11 months ago
- 该代码与B站上的视频 https://www.bilibili.com/video/BV18L41197Uz/?spm_id_from=333.788&vd_source=eefa4b6e337f16d87d87c2c357db8ca7 相关联。☆66Updated last year
- 多模态 MM +Chat 合集☆249Updated last month
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆26Updated last year
- Using pattern matcher in onnx model to match and replace subgraphs.☆77Updated last year
- A Light-Weight Framework for Open-Set Object Detection with Decoupled Feature Alignment in Joint Space☆78Updated 2 months ago
- ☆50Updated 5 months ago
- 使用 CUDA C++ 实现的 llama 模型推理框架☆48Updated 4 months ago
- Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeed☆82Updated 5 months ago
- Official code for paper: [CLS] Attention is All You Need for Training-Free Visual Token Pruning: Make VLM Inference Faster.☆59Updated 3 months ago
- LLM101n: Let's build a Storyteller 中文版☆130Updated 7 months ago
- llm-export can export llm model to onnx.☆272Updated 2 months ago
- Enhance LLM agents with rich tool APIs☆382Updated 6 months ago