LLM Inference Engine: High-performance CUDA-accelerated framework for large language model inference A cutting-edge, open-source implementation of a large language model (LLM) inference engine, optimized for consumer-grade hardware. This project showcases advanced techniques in GPU acceleration, memory management, and algorithmic optimizations
☆11Sep 29, 2024Updated last year
Alternatives and similar repositories for LlamaInfer
Users that are interested in LlamaInfer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Using gemini AI and VLMs to control a robot in simulation☆13Jan 3, 2025Updated last year
- This is the combined collection of the course notes for some of the computer science classes at CMU released online.☆66Jan 20, 2025Updated last year
- This program captures an image from a Linux video device (/dev/video0) and then displays and saves it using OpenCV4.☆13Feb 2, 2022Updated 4 years ago
- Lightweight Llama 3 8B Inference Engine in CUDA C☆54Mar 21, 2025Updated last year
- TurtleSim A* Path Planning with Meta LLaMA-3.1-405B-Instruct model powered by NVIDIA / LlamaIndex Agent☆13Oct 14, 2024Updated last year
- 🌟 从LLaMA2开启大语言模型原理与实践教程☆76Oct 29, 2025Updated 4 months ago
- ☆25May 25, 2024Updated last year
- ☆16Aug 26, 2025Updated 6 months ago
- Detecting Sequence Signals in Targeting Peptides Using Deep Learning☆14Sep 5, 2019Updated 6 years ago
- Tutorial to start working with Multiple Instance Learning☆14Jul 5, 2023Updated 2 years ago
- Pretrain、Posttrain、RAG、Agent等大模型相关的基础项目合集☆32Dec 7, 2025Updated 3 months ago
- sim_llm 是一个基于 ROS2 的仿真测试，使用当下热门的大语言模型控制 turtle 做出一些简单的行动。☆21May 12, 2025Updated 10 months ago
- a simple tool for real-time monitoring video and summarization with LLMs☆32Mar 2, 2026Updated 3 weeks ago
- ☆11Mar 6, 2026Updated 2 weeks ago
- Free knowledge base theme material design☆13Feb 11, 2020Updated 6 years ago
- The first multimodal QA dataset specifically designed for evaluating large TCM language models.☆21Oct 24, 2025Updated 4 months ago
- EnzyMM - Enzyme Motif Miner - Geometric matching of catalytic motifs in protein structures.☆38Updated this week
- This is the source code of CCDiff, a novel structure-guided diffusion framework to address the challenges of generating realistic and con…☆26Nov 13, 2025Updated 4 months ago
- Residue Level Alignment☆22Nov 21, 2024Updated last year
- 面向可信执行环境的OS。☆12May 9, 2025Updated 10 months ago
- code for Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Princip…☆23Jul 26, 2025Updated 7 months ago
- It's an implementation of Efros and Freeman's "Image Quilting and Texture Synthesis" 2001☆18Jul 2, 2019Updated 6 years ago
- Code for "Attentive Variational Information Bottleneck for TCR-peptide Interaction Prediction", Grazioli et al., Bioinformatics 2022☆23Jan 10, 2023Updated 3 years ago
- Another dynamically-typed, lightweight programming language☆12May 5, 2015Updated 10 years ago
- ShanghaiTech CS110 (Computer Architecture I), Spring 2022☆11Dec 14, 2023Updated 2 years ago
- 毕业设计：视频监控系统， qt + v4l2 + opencv + sqlite☆28May 5, 2021Updated 4 years ago
- MagickCache is a secure, high-performance caching tool for images, videos, audio, and metadata. It uses memory mapping for fast access, s…☆19Feb 23, 2026Updated last month
- A 32 point radix-2 FFT module written in Verilog☆25Jun 28, 2020Updated 5 years ago
- ☆22May 14, 2025Updated 10 months ago
- PhysFS fork with magic streams☆12Apr 4, 2019Updated 6 years ago
- Trajectory Optimization-based control of autonomous vehicle following Bicycle model dynamics☆27Aug 30, 2020Updated 5 years ago
- EmbodiedAgents is a fully-loaded ROS2 based framework for creating interactive physical agents that can understand, remember, and act upo…☆50Mar 17, 2026Updated last week
- OpenGL rendering engine with software configured pipeline☆10Apr 9, 2025Updated 11 months ago
- 根据正则表达式生成其对应 DFA 的状态转移图☆15Nov 20, 2018Updated 7 years ago
- ☆10Sep 7, 2022Updated 3 years ago
- Using python poetry on Google Colab☆17Feb 20, 2023Updated 3 years ago
- 使用强化学习训练PPT的Agent☆68Oct 16, 2025Updated 5 months ago
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆26Nov 11, 2024Updated last year
- 带RL 训练50000timesteps☆21Apr 14, 2021Updated 4 years ago