A high-throughput and memory-efficient inference and serving engine for LLMs
☆11Sep 4, 2025Updated 10 months ago
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆11Oct 11, 2023Updated 2 years ago
- ☆12Jul 22, 2026Updated last week
- ☆14Dec 22, 2023Updated 2 years ago
- Inference Llama 2 in one file of pure Haskell (A port of llama2.c from Andrej Karpathy)☆14Oct 17, 2025Updated 9 months ago
- [EMNLP 2022] RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees☆11Jul 15, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- arXiv-Chat: An AI research assistant and Discord bot☆13Jul 16, 2023Updated 3 years ago
- This Repo Contains Script To Fine Tune Open Source Models Using Unsloth by using UI with simple click and progress☆13Oct 3, 2024Updated last year
- Code for data reduction and analysis of Galaxy Zoo 2☆14May 20, 2016Updated 10 years ago
- The Swift Programming Language☆12Updated this week
- Port of Facebook's LLaMA model in C/C++☆13Updated this week
- ☆13Oct 25, 2024Updated last year
- Android wrapper for Inference Llama 2 in one file of pure C☆18Aug 21, 2023Updated 2 years ago
- The Swift Programming Language☆19Jan 8, 2021Updated 5 years ago
- Inference Llama 2 in one file of pure Cuda☆17Aug 20, 2023Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- a temporal graph analytics library based on Flink Stateful Functions☆12Jun 8, 2023Updated 3 years ago
- Inference Llama 2 in one file of pure C☆13Nov 17, 2023Updated 2 years ago
- ☆12Dec 19, 2024Updated last year
- Unofficial reimplementation of ViR: Vision Retention Networks by Hatamizadeh et. al. (https://arxiv.org/abs/2310.19731)☆18Jul 26, 2024Updated 2 years ago
- open-source visual bookmark manager built with next.js. simple, and easy to set up☆26Nov 23, 2024Updated last year
- Template repo for Python projects, especially those focusing on machine learning and/or deep learning.☆15Jan 14, 2026Updated 6 months ago
- 蚂蚁开源技术沙龙演讲材料归档☆29Jun 16, 2026Updated last month
- Docker packaging for Apache Flink Stateful Functions☆18May 15, 2026Updated 2 months ago
- multilabel categorical crossentropy☆15Apr 26, 2020Updated 6 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Manages vllm-nccl dependency☆18Jun 3, 2024Updated 2 years ago
- Deep learning examples for the Instant Super Computer☆20Jan 28, 2026Updated 6 months ago
- ☆13Feb 22, 2023Updated 3 years ago
- Model implementation and explorative UI for the paper "Towards Cost-Optimal Query Processing in the Cloud". Slides: https://bit.ly/37ZfeP…☆17Sep 17, 2025Updated 10 months ago
- Distributed consensus system with Map interface based on Apache Ratis☆28Nov 27, 2023Updated 2 years ago
- every 20 minutes, look 20 feet away for 20 seconds, kedip keeps track so you don't have to☆21Apr 26, 2026Updated 3 months ago
- Dockerfile examples for reproducing package cache (e.g., `/etc/apk/cache`)☆30Sep 16, 2023Updated 2 years ago
- A Benchmark Harness for Systematic and Robust Evaluation of Streaming State Stores☆17Apr 24, 2024Updated 2 years ago
- This is the repository for TimelineQA, a benchmark for querying lifelogs.☆27Jul 5, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 针对qwen微调模型进行数据预处理☆13Jan 8, 2024Updated 2 years ago
- the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly☆33Oct 19, 2024Updated last year
- PostText is a QA system for querying your text data. When appropriate structured views are in place, PostText is good at answering querie…☆32Jun 14, 2023Updated 3 years ago
- Javascript / node.js code to read FCS flow cytometry data☆19Jun 19, 2023Updated 3 years ago
- Language Models as Multi-Modal Query Planners☆20Mar 20, 2024Updated 2 years ago
- Inference Llama 2 in one file of pure C. Nahh wait, now fresh in Julia!☆25Aug 2, 2023Updated 2 years ago
- GitHub Action for Mage☆34Updated this week