AI Infra LLM infer/ tensorrt-llm/ vllm
☆24Jun 7, 2026Updated last month
Alternatives and similar repositories for llm-deploy
Users that are interested in llm-deploy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Deep insight tensorrt, including but not limited to qat, ptq, plugin, triton_inference, cuda☆23Jul 2, 2026Updated last week
- Principles and Methodologies for Serial Performance Optimization (OSDI' 25)☆30Jun 5, 2025Updated last year
- LLM Agents: Landing Page Generation for an E-commerce Platform Using CrewAI, Groq-LangChain and Qdrant☆15May 30, 2024Updated 2 years ago
- ☆20Jun 1, 2023Updated 3 years ago
- introduce AI infra knowledges. 人工智能系统基础架构知识库☆16Jun 4, 2023Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- 📦 eCAL C/C++ runtime core☆12Oct 21, 2024Updated last year
- 用大模型批量处理数据，现支持各种大模型做OCR，支持通义千问, 月之暗面, 百度飞桨OCR, OpenAI 和LLAVA。Use LLM to generate or clean data for academic use. Support OCR with qwen, m…☆17Sep 15, 2024Updated last year
- A High-Level DRAM Timing, Power and Area Exploration Tool☆31Jul 29, 2020Updated 5 years ago
- 基于 python3+ django2.2的前后端分离模板☆11Nov 22, 2022Updated 3 years ago
- [JCheminfo] Ualign: pushing the limit of template-free retrosynthesis prediction with unsupervised SMILES alignment☆18May 10, 2026Updated last month
- B-Roll: Video data in rosbag2 plugins and utilities☆12Nov 19, 2025Updated 7 months ago
- Flexible and Pluggable Serving Engine for Diffusion LLMs☆143Updated this week
- 使用 python、Latex 绘制神经网络架构图。☆20Jun 2, 2022Updated 4 years ago
- SimplePIM is the first high-level programming framework for real-world processing-in-memory (PIM) architectures. Described in the PACT 20…☆36Oct 23, 2023Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆183Jul 12, 2024Updated last year
- A curated collection of projects, benchmarks, and research papers focused on reproducing and advancing the DeepSeek R1 framework.☆15Mar 19, 2025Updated last year
- Simple example application to show how to allocate dmabufs from user space (from a dmabuf heap) and use them for v4l2 capture.☆16Aug 2, 2023Updated 2 years ago
- Examples of using Isaac ROS GEMs together☆19May 1, 2026Updated 2 months ago
- Tensorflow code for WACV 2019 paper "Attention Based Natural Language Grounding by Navigating Virtual Environment" - https://arxiv.org/ab…☆17Nov 7, 2018Updated 7 years ago
- [COLM 2024] SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models☆24Oct 5, 2024Updated last year
- Provides a demo of micro-ROS based on ST Disco L475 IOT01 board.☆13Jul 14, 2021Updated 4 years ago
- ☆28Sep 9, 2025Updated 10 months ago
- ros2_v4l2_camera with shm_msg based zero-copy and more!☆14Dec 8, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆14Jun 20, 2025Updated last year
- RMW implementation for EmbeddedRTPS middleware☆16Apr 21, 2023Updated 3 years ago
- learning materials of driveos from nvidia drive sdk.☆13Jun 10, 2026Updated 3 weeks ago
- Rust wrapper for Eclipse iceoryx™ - true zero-copy inter-process-communication☆14Sep 14, 2021Updated 4 years ago
- ROS2 package that allows recording without interprocess communication☆19Apr 16, 2026Updated 2 months ago
- ROS2 image transport plugin using libav(ffmpeg) for generating foxglove CompressedVideo messages☆12Jun 29, 2026Updated last week
- ☆13Sep 10, 2025Updated 9 months ago
- ROS Graph Monitor - libraries and nodes to check the ROS application structure as a whole☆36Jun 2, 2026Updated last month
- GeoJSON for Modern C++☆16Mar 22, 2026Updated 3 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Depict GPU memory footprint during DNN training of PyTorch☆11Nov 17, 2022Updated 3 years ago
- Abstract 2D multi-robot multi-interface mobile robot simulator (fork of MobileSim simulator with various fixes, optimizations, and added …☆16May 30, 2023Updated 3 years ago
- Fast command-line tool to extract useful data from LTTng traces of ROS applications.☆21May 13, 2026Updated last month
- ROS 2 Type adapters for common C++ frameworks☆18Mar 26, 2025Updated last year
- Bridge Autoware and Carla with Zenoh☆20Jun 9, 2026Updated last month
- A lite version based on cyber RT☆14Feb 5, 2026Updated 5 months ago
- ☆14Apr 20, 2022Updated 4 years ago