LLM Inference Engine: High-performance CUDA-accelerated framework for large language model inference A cutting-edge, open-source implementation of a large language model (LLM) inference engine, optimized for consumer-grade hardware. This project showcases advanced techniques in GPU acceleration, memory management, and algorithmic optimizations
☆11Sep 29, 2024Updated 2 years ago
Alternatives and similar repositories for LlamaInfer
Users that are interested in LlamaInfer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Using gemini AI and VLMs to control a robot in simulation☆12Jan 3, 2025Updated last year
- ☆10Nov 25, 2022Updated 3 years ago
- This is the combined collection of the course notes for some of the computer science classes at CMU released online.☆65Jan 20, 2025Updated last year
- This program captures an image from a Linux video device (/dev/video0) and then displays and saves it using OpenCV4.☆13Feb 2, 2022Updated 4 years ago
- TurtleSim A* Path Planning with Meta LLaMA-3.1-405B-Instruct model powered by NVIDIA / LlamaIndex Agent☆13Oct 14, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆16Aug 26, 2025Updated last year
- ☆16May 13, 2022Updated 4 years ago
- SGLang（高性能LLM服务框架）的深度学习资料☆16Sep 25, 2025Updated last year
- ☆10Oct 20, 2024Updated 2 years ago
- ☆12Apr 25, 2024Updated 2 years ago
- ☆14Oct 13, 2026Updated 2 weeks ago
- Point cloud pre-process for PointNetGPD & Kinect v2☆13Mar 24, 2021Updated 5 years ago
- robot grasp task for JCAR competition☆11Oct 12, 2019Updated 7 years ago
- Code for experimentation of "Protocol Reverse Engineering Using Neural Networks"☆12Jun 13, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Web app to annotate images for surgical tooltip localisation.☆15May 27, 2022Updated 4 years ago
- ☆19Nov 7, 2020Updated 5 years ago
- ☆14Apr 4, 2023Updated 3 years ago
- code for Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Princip…☆24Jul 26, 2025Updated last year
- ☆19May 2, 2020Updated 6 years ago
- Detecting Sequence Signals in Targeting Peptides Using Deep Learning☆14Sep 5, 2019Updated 7 years ago
- ☆13Oct 9, 2026Updated 3 weeks ago
- A simple GUI for running ForceBindIP☆18Nov 30, 2024Updated last year
- Free knowledge base theme material design☆13Feb 11, 2020Updated 6 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- 面向可信执行环境的OS。☆12May 9, 2025Updated last year
- Representation learning for protein functional site analysis☆17Aug 11, 2023Updated 3 years ago
- Protein Annotation by Residue-Specific Enrichment: a site-based statistical method for interpretable protein function annotation☆16Jun 16, 2025Updated last year
- PhysFS fork with magic streams☆12Apr 4, 2019Updated 7 years ago
- ☆16May 22, 2024Updated 2 years ago
- sim_llm 是一个基于 ROS2 的仿真测试，使用当下热门的大语言模型控制 turtle 做出一些简单的行动。☆22May 12, 2025Updated last year
- Protein Fold Classification at Scale: Benchmarking and Pretraining (ICML 2026 oral)☆21May 24, 2026Updated 5 months ago
- A 32 point radix-2 FFT module written in Verilog☆25Jun 28, 2020Updated 6 years ago
- SNARE Dataset with MATCH and LaGOR models☆23Mar 27, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ShanghaiTech CS110 (Computer Architecture I), Spring 2022☆11Dec 14, 2023Updated 2 years ago
- Real-time camera, timelapse, and LLM summarization system for IP camera and NVR workflows☆34Oct 20, 2026Updated last week
- ☆27May 25, 2024Updated 2 years ago
- Codes of CVPR2022 paper: Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction☆32Aug 23, 2022Updated 4 years ago
- 根据正则表达式生成其对应 DFA 的状态转移图☆15Nov 20, 2018Updated 7 years ago
- 本项目成功设计并实现了一个基于RAG架构的智能法律问答系统，具备完善的用户管理、多格式文档解析、智能问答交互、知识库动态管理等一系列核心功能。该系统通过API接入DeepSeek-R1/Lora训练的DeepSeek-R1-Distill-Qwen-1.5B作为大语言模型，…☆20Nov 5, 2025Updated 11 months ago
- 毕业设计：视频监控系统， qt + v4l2 + opencv + sqlite☆27May 5, 2021Updated 5 years ago