NetEase-Media / grps
【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架,支持dynamic batching、streaming模式,支持python/c++双语言,可限制,可拓展,高性能。帮助用户快速地将模型部署到线上,并通过http/rpc接口方式提供服务。
☆144Updated 2 weeks ago
Related projects: ⓘ
- 【grps接入trtllm】通过接入TensorRT-LLM以及Tokenizers.cpp实现纯c++版本高性能LLM服务,兼容OpenAI接口协议,支持chat和function call模式。☆40Updated last week
- 使用deepspeed从头开始训练一个LLM,经过pretrain和sft阶段,验证llm学习知识、理解语言、回答问题的能力☆145Updated 2 months ago
- TengineGst is a streaming media analytics framework, based on GStreamer multimedia framework, for creating varied complex media analytics…☆74Updated 2 years ago
- ☆52Updated this week
- 🚀 Do not need libtorch, pure C++ TensorRT deploys SOLOv2 etc, which can be quickly ported to NX/TX2.☆50Updated 2 years ago
- Algorithm acceleration landing framework, let you complete the development of algorithm at low cost.eg: Facedetect, FaceLandmark..☆90Updated 3 years ago
- Inference of superpoint feature extraction with pure C/C++☆34Updated 6 months ago
- 模型部署白皮书(CUDA|ONNX|TensorRT|C++)🚀🚀🚀☆173Updated 3 weeks ago
- Ai edge toolbox,专门面向边端设备尤其是嵌入式RTOS平台,AI模型部署工具链,包括模型推理引擎和模型压缩工具☆162Updated 9 months ago
- A Tiny structure of pytorch for learning; 一个最小pytorch的实现☆46Updated 2 months ago
- 教你只用最基本的python语法和numpy一步步实现深度学习框架☆117Updated last month
- SegmentAnything-OnnxRunner is an example using Meta AI Research's SAM onnx model in C++.The encoder and decoder of SAM are decoupled in t…☆94Updated 9 months ago
- 本项目使用YOLOv4模型,并在对数字信号灯进行数字识别时采用opencv算法。☆122Updated last year
- GAL-DAWN: An Novel High performance computing Library of Graph Algorithms based on DAWN, CUDA/C++☆116Updated last month
- This tool(enhance_long) aims to enhance the LlaMa2 long context extrapolation capability in the lowest-cost approach, preferably without …☆47Updated 9 months ago
- 实体关系联合抽取☆175Updated 4 months ago
- 本项目旨在结合以往研究人员的代表性工作,从多个维度评估sft数据,并自动化过滤sft数据。☆55Updated 6 months ago
- Multilingual Retrieval on Yelp Search Engine ⚡☆102Updated 2 years ago
- ☆60Updated last year
- 莫甘娜问卷表单编辑器,低代码快速搭建表单,AI表单生成,表单数据搜集统计☆203Updated last month
- ☆73Updated this week
- Build you own Social Apps like facebook twitter etc. 使用kotlin和React来搭建一个社交apps,类似小红书,微博☆132Updated 2 months ago
- Ocean是使用基于大语言模型文本知识向量抽取和Transformer构建的校园开源文库☆19Updated 5 months ago
- Ein multimodaler, multi-intelligenter Entwicklungsrahmen☆56Updated this week
- Geniz interactive code-gen tool☆141Updated 3 months ago
- Harnessing the Power of AI to Navigate the Information Age – Uncovering Truth, Promoting Transparency, and Championing Fact-Based Discour…☆210Updated last year
- Accelerate your Stable Diffusion inference with the library's universal C/C++ framework design, powered by ONNXRuntime & across platforms…☆610Updated last month
- ☆127Updated this week
- An ultra fast tiny model for lane detection, using onnx_parser, TensorRTAPI, torch2trt to accelerate. our model support for int8, dynamic…☆141Updated 3 years ago
- AutoAnys is an innovative, open-source Robotic Process Automation (RPA) platform designed to revolutionize the automation landscape. Buil…☆94Updated 2 months ago