Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.
☆23Apr 12, 2024Updated 2 years ago
Alternatives and similar repositories for Efficiently-Serving-LLMs
Users that are interested in Efficiently-Serving-LLMs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Minimal tensor computation framework in pure Go with SIMD assembly, inspired by tinygrad☆17Oct 11, 2026Updated last week
- Behaviors for 99☆14Jan 16, 2026Updated 9 months ago
- ☆13Jul 22, 2023Updated 3 years ago
- Deploy, launch and use LLMs on AWS☆16Jun 2, 2023Updated 3 years ago
- Deploy SageMaker models with Terraform☆23Feb 14, 2018Updated 8 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Truss for deploying Starcoder to Baseten or other platforms☆12Jun 15, 2023Updated 3 years ago
- A course offered by Stanford University☆16Sep 4, 2020Updated 6 years ago
- MoJev: typed, calibrated decisions in one forward pass.☆30Sep 23, 2026Updated 3 weeks ago
- A distributed event processing pipeline built in Go☆27Jul 28, 2026Updated 2 months ago
- This project explores training data extraction attacks on the LLaMa 7B, GPT-2XL, and GPT-2-IMDB models to discover memorized content usin…☆15Jun 15, 2023Updated 3 years ago
- minimal implementation of sft with gpt2-124M☆16Nov 10, 2025Updated 11 months ago
- Tutorial for LLM developers about engine design, service deployment, evaluation/benchmark, etc. Provide a C/S style optimized LLM inferen…☆19Sep 5, 2023Updated 3 years ago
- Nvim plugin for reviewing PRs with side by side diffs, inline comments, and persistent review state.☆20Apr 3, 2026Updated 6 months ago
- ☆10Aug 18, 2021Updated 5 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Repo for the Advanced Python Skills course that I created (hosted in Udemy and Skillshare)☆16Nov 1, 2020Updated 5 years ago
- ☆25May 9, 2022Updated 4 years ago
- Unlock automation and system building with the ChatGPT API. Master chain calls, Python interactions, and create a customer service chatbo…☆19Jun 21, 2023Updated 3 years ago
- Batteries-included comment plugin for neovim with supports of line comment, block comment, textobjects, real sticky cursor and more!☆44Oct 14, 2026Updated last week
- AI Software Bill of Materials for EU AI Act☆12Jan 18, 2024Updated 2 years ago
- ☆10Aug 24, 2023Updated 3 years ago
- ☆13Sep 22, 2024Updated 2 years ago
- Code repository for World Model From Scratch book☆38Oct 1, 2026Updated 3 weeks ago
- Implementation of the G-CORE graph query language on Spark☆15Aug 25, 2021Updated 5 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parall…☆24Aug 11, 2026Updated 2 months ago
- DeepLearning.ai course: Knowledge Graphs for RAG☆22Jun 16, 2024Updated 2 years ago
- COMET for African languages☆11Jan 24, 2025Updated last year
- For my IBM Data Science Professional certificate capstone project in early 2020, I used pandas, the FourSquare API, Folium, and other Pyt…☆13Dec 31, 2020Updated 5 years ago
- You’ll explore new advancements like ChatGPT’s function calling capability, and build a conversational agent using a new syntax called La…☆16Oct 28, 2023Updated 2 years ago
- EnriCo: Enriched Representation and Globally Constrained Inference for Entity and Relation Extraction☆27May 22, 2024Updated 2 years ago
- Community detection in complex networks using hybrid quantum annealing on Amazon Braket☆13Jul 6, 2023Updated 3 years ago
- A Python package that helps you create paths relative to the project root☆15Dec 27, 2022Updated 3 years ago
- Repository for React Fundamentals classroom demonstration contacts app☆11Nov 19, 2024Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆13Oct 25, 2024Updated last year
- The minimal, ad-hoc way of plug and play NebulaGraph with pip install, even inside Colab Notebook!☆21May 24, 2024Updated 2 years ago
- This project scrapes text from Telugu books(Novels)☆11Aug 3, 2021Updated 5 years ago
- Nebula docker image for development☆16Apr 1, 2026Updated 6 months ago
- Companion repository for the Orchestrating Workflows for GenAI Applications course on Deeplearning.AI: https://bit.ly/45P4WQN☆34Jun 11, 2025Updated last year
- This repository will take you through creating a FastAPI StableDiffusion app (including Dockerfile) all the way to adding a new feature u…☆38Nov 9, 2022Updated 3 years ago
- Code showing how to use a model based on the ML model base class.☆10Sep 30, 2022Updated 4 years ago