Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.
☆23Apr 12, 2024Updated 2 years ago
Alternatives and similar repositories for Efficiently-Serving-LLMs
Users that are interested in Efficiently-Serving-LLMs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This is the code for neural-Jacana aligner, and the data for MultiMWA dataset.☆20Feb 12, 2023Updated 3 years ago
- Deploy SageMaker models with Terraform☆23Feb 14, 2018Updated 8 years ago
- Official Github Repository for "Spectral-Risk Safe Reinforcement Learning with Convergence Guarantees". (NeurIPS 2024)☆11Nov 30, 2025Updated 9 months ago
- This project explores training data extraction attacks on the LLaMa 7B, GPT-2XL, and GPT-2-IMDB models to discover memorized content usin…☆15Jun 15, 2023Updated 3 years ago
- Tutorial for LLM developers about engine design, service deployment, evaluation/benchmark, etc. Provide a C/S style optimized LLM inferen…☆19Sep 5, 2023Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- AI Software Bill of Materials for EU AI Act☆12Jan 18, 2024Updated 2 years ago
- Code space for L4DC paper "State-wise Safe Reinforcement Learning With Pixel Observations"☆11Apr 5, 2024Updated 2 years ago
- OpenProbe is an advanced agent-based search system that performs deep web searches to answer complex questions. It uses a graph-based app…☆23Updated this week
- ☆25May 9, 2022Updated 4 years ago
- oneNeuron Pytroch basics course docs plus code☆10Mar 14, 2022Updated 4 years ago
- ☆15Aug 7, 2025Updated last year
- The Chia Network Nebula Graph database Importer☆11Jan 16, 2023Updated 3 years ago
- ☆10Aug 24, 2023Updated 3 years ago
- Implementation of the G-CORE graph query language on Spark☆15Aug 25, 2021Updated 5 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- [ICRA'24] Influence of Camera-LiDAR Configuration on 3D Object Detection for Autonomous Driving☆21Sep 14, 2024Updated 2 years ago
- COMET for African languages☆11Jan 24, 2025Updated last year
- ☆36Jun 8, 2026Updated 3 months ago
- LLM中相关RLHF算法实现与学习☆15Apr 13, 2025Updated last year
- A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning☆17Jul 20, 2020Updated 6 years ago
- LLM Evals Leaderboard☆49Nov 21, 2023Updated 2 years ago
- ☆20Jun 15, 2023Updated 3 years ago
- A python program which learns to recognize an unknown face using Principal Component Analysis (PCA).☆12Jun 29, 2017Updated 9 years ago
- You’ll explore new advancements like ChatGPT’s function calling capability, and build a conversational agent using a new syntax called La…☆16Oct 28, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Simple PyTorch implementation of NeRF (Neural Radiance Fields).☆20May 11, 2022Updated 4 years ago
- Nebula docker image for development☆16Apr 1, 2026Updated 5 months ago
- Community detection in complex networks using hybrid quantum annealing on Amazon Braket☆13Jul 6, 2023Updated 3 years ago
- Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 30+ benchmarks☆15Feb 17, 2025Updated last year
- Official code of our work, VCSR: Mutable CSR Graph Format Using Vertex-Centric Packed Memory Array [CCGrid 2022].☆15Jun 30, 2022Updated 4 years ago
- A Java library for quantum programming using Quil.☆16Jul 23, 2018Updated 8 years ago
- This repository will take you through creating a FastAPI StableDiffusion app (including Dockerfile) all the way to adding a new feature u…☆38Nov 9, 2022Updated 3 years ago
- Code showing how to use a model based on the ML model base class.☆10Sep 30, 2022Updated 3 years ago
- 一个带登录界面的基于MINST的花卉识别系统☆15Apr 17, 2021Updated 5 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- data related codebase for polyglot project☆19Mar 30, 2023Updated 3 years ago
- my wezterm conf☆20Dec 6, 2024Updated last year
- Encountering 14 different Naive RAG fails and using KG to solve it☆33Dec 4, 2025Updated 9 months ago
- 提供B站倍速播放功能，支持鼠标滚轮调节、触控板调节、自定义记忆播放速度。Provides customize playback speed options for Bilibili.☆22Nov 24, 2025Updated 10 months ago
- Official implementation of the paper: Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificate☆25Dec 4, 2023Updated 2 years ago
- Exercises for the CERN Openlab GPU lecture☆15Aug 5, 2026Updated last month
- An open API providing access to the UNESCO World Heritage Site data.☆14Jan 19, 2023Updated 3 years ago