There are many articles that cover the principles of reducing latency optimization for LLMs, however it is often unclear how to actually implement these principles. This repository provides practical techniques for reducing the latency of GenAI applications.
☆37May 6, 2024Updated 2 years ago
Alternatives and similar repositories for The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications
Users that are interested in The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆14Apr 1, 2025Updated last year
- Generative AI Ops RAG project template☆44Apr 21, 2026Updated 2 months ago
- This hands-on walks you through fine-tuning an open source LLM on Azure and serving the fine-tuned model on Azure. It is intended for Dat…☆12Jun 23, 2024Updated 2 years ago
- This solution converts speech to text and then processes and summarizes the text based on the prompt scenario.☆20Aug 8, 2024Updated last year
- ☆11Mar 19, 2020Updated 6 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Solution Accelerator: Using Logic Apps & Form Recognizer☆15Sep 22, 2023Updated 2 years ago
- Deep Learning for Natural Language Processing (NLP) using Variational Autoencoders (VAE)☆11May 15, 2021Updated 5 years ago
- Tui Utility to test REST APIs☆13Nov 20, 2023Updated 2 years ago
- An R package to help assess the sensitivity of a Bayesian model (fitted with Stan) to the specification of its likelihood and priors☆11May 29, 2026Updated last month
- ☆14Oct 30, 2021Updated 4 years ago
- This solution demonstrates the setup and deployment of Amazon SageMaker Studio into a private VPC and implementation of multi-layer secur…☆22Jul 31, 2022Updated 3 years ago
- Creates an Azure AI Studio hub, project and required dependent resources including Azure Open AI Service, Cognitive Search and more.☆33Oct 2, 2024Updated last year
- This lab is a 1-day/2-day end-to-end SLM workshop led and developed by AI GBB. Attendees will learn how to quickly and easily perform the…☆46Jan 22, 2026Updated 5 months ago
- Assistant API to chat with tabular data and perform analytics in natural language.☆57Aug 30, 2024Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- GAN-Based Text Generation☆14Apr 19, 2020Updated 6 years ago
- Azure OpenAI benchmarking tool☆29Apr 4, 2025Updated last year
- Convert any image into a Region Adjacency Graph (RAG)☆12Apr 27, 2020Updated 6 years ago
- Library to convert natural language utterance into a structured domain specific language☆20Feb 11, 2026Updated 4 months ago
- Machine Learning Projects for the Dr. Andrew Ng's course on Coursera☆12Dec 19, 2017Updated 8 years ago
- An offline exercise for Data Analyst candidates☆12Jun 28, 2022Updated 4 years ago
- Indonesian law dataset containing section annotation of court decision documents☆19Jul 7, 2022Updated 3 years ago
- A series of templates for a Quarto project to convert a single Markdown input into beautiful and simple Word, HTML, and PDF worksheets.☆14Feb 12, 2022Updated 4 years ago
- This project is uploaded as a demo to showcase various components and patterns☆11Oct 23, 2017Updated 8 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- 🚀 Embark on your agentic journey !☆29May 28, 2025Updated last year
- ☆15Mar 6, 2024Updated 2 years ago
- ☆125Jun 29, 2026Updated last week
- The Atlas of Pidgin and Creole Language Structures☆16Nov 9, 2022Updated 3 years ago
- ☆11Nov 20, 2020Updated 5 years ago
- This solution converts speech to text and then processes and summarizes the text based on the prompt scenario.☆39Oct 8, 2024Updated last year
- Contoso Outdoors Company web application shown at Microsoft Ignite☆59May 10, 2024Updated 2 years ago
- LaTeX macros to fill in PDF.☆13Jan 31, 2019Updated 7 years ago
- High-performance open-source orchestration utility that utilizes EBS Direct APIs to efficiently clone, copy and migrate EBS snapshots to …☆39Dec 11, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆15Aug 30, 2021Updated 4 years ago
- Azure OpenAI benchmarking tool☆152May 28, 2024Updated 2 years ago
- Open source repository to help others learn about IaC and the various flavors☆18Apr 16, 2024Updated 2 years ago
- ☆30Apr 8, 2022Updated 4 years ago
- 🔒 Reference MCP servers that demo how authentication works with the current Model Context Protocol spec.☆50Jan 7, 2026Updated 5 months ago
- Document classification with Apache Spark on an American Classic☆10Sep 25, 2015Updated 10 years ago
- UM1 test programs and sample code☆11Jul 25, 2022Updated 3 years ago