AI Infra LLM infer/ tensorrt-llm/ vllm
☆24Jul 26, 2026Updated this week
Alternatives and similar repositories for llm-deploy
Users that are interested in llm-deploy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Deep insight tensorrt, including but not limited to qat, ptq, plugin, triton_inference, cuda☆24Updated this week
- 📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉☆21Mar 30, 2025Updated last year
- FastSAM 部署版本，便于移植不同平，部署简单、运行速度快。☆25May 30, 2024Updated 2 years ago
- 基于电商导购机器人，自然语言理解（NLU），文本纠错，歧义词消歧☆12May 5, 2020Updated 6 years ago
- Implementation example of Distributed Tensorflow☆10Jul 22, 2017Updated 9 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 「城语」APP基于A级景区、历史古迹、文物保护单位等基础数据，利用先进的大模型能力实现智能化的Citywalk 路线规划，包括设计一条路线、生成路线攻略、生成景点的推荐理由等三大核心功能；利用大模型减少了人工编辑和推荐的工作量，并可以根据游客的需求进行个性化定制，提升了游客…☆19Feb 20, 2024Updated 2 years ago
- ☆11Jan 9, 2019Updated 7 years ago
- LLM Agents: Landing Page Generation for an E-commerce Platform Using CrewAI, Groq-LangChain and Qdrant☆15May 30, 2024Updated 2 years ago
- A GPU accelerated error-bounded lossy compression for scientific data.☆100Jul 2, 2026Updated 3 weeks ago
- introduce AI infra knowledges. 人工智能系统基础架构知识库☆17Jun 4, 2023Updated 3 years ago
- ☆11Feb 13, 2025Updated last year
- ☆20Jun 1, 2023Updated 3 years ago
- The ComponentContainer and Executor that assign a dedicated thread for each callback group.☆10Jun 20, 2025Updated last year
- ☆17Jun 10, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- [JCheminfo] Ualign: pushing the limit of template-free retrosynthesis prediction with unsupervised SMILES alignment☆18May 10, 2026Updated 2 months ago
- B-Roll: Video data in rosbag2 plugins and utilities☆12Nov 19, 2025Updated 8 months ago
- 用大模型批量处理数据，现支持各种大模型做OCR，支持通义千问, 月之暗面, 百度飞桨OCR, OpenAI 和LLAVA。Use LLM to generate or clean data for academic use. Support OCR with qwen, m…☆17Sep 15, 2024Updated last year
- Ament task provider extension for vscode☆10Updated this week
- deepseek思维树模式实现☆23Jul 17, 2025Updated last year
- ☆17May 25, 2025Updated last year
- Visualize the Expert Parallelism Load Balancer☆19Mar 15, 2025Updated last year
- Repository for the COLM 2025 paper SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths☆19Jul 10, 2025Updated last year
- ☆14Aug 15, 2025Updated 11 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Codebase of Truncated Consistency Models (ICLR 2025)☆34Jan 24, 2025Updated last year
- ☆12Jul 15, 2026Updated 2 weeks ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆184Jul 12, 2024Updated 2 years ago
- ONNX Python Examples☆16Sep 13, 2022Updated 3 years ago
- Provides a demo of micro-ROS based on ST Disco L475 IOT01 board.☆13Jul 14, 2021Updated 5 years ago
- Isolates any given process using the unshare system call. Suited for ROS, though can work for any process.☆13Jan 31, 2024Updated 2 years ago
- zlai☆23Sep 29, 2024Updated last year
- [COLM 2024] SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models☆24Oct 5, 2024Updated last year
- ros2_v4l2_camera with shm_msg based zero-copy and more!☆14Dec 8, 2022Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- SimplePIM is the first high-level programming framework for real-world processing-in-memory (PIM) architectures. Described in the PACT 20…☆36Oct 23, 2023Updated 2 years ago
- PiCAS executor + ROS 2 Real-Time Working Group's reference system☆12Oct 4, 2023Updated 2 years ago
- ROS2 image transport plugin using libav(ffmpeg) for generating foxglove CompressedVideo messages☆14Jun 29, 2026Updated last month
- ☆13Sep 10, 2025Updated 10 months ago
- learning materials of driveos from nvidia drive sdk.☆13Jun 10, 2026Updated last month
- Rust wrapper for Eclipse iceoryx™ - true zero-copy inter-process-communication☆14Sep 14, 2021Updated 4 years ago
- aiml的C++实现，支持中文，改自libaiml☆16Dec 2, 2012Updated 13 years ago