There are many articles that cover the principles of reducing latency optimization for LLMs, however it is often unclear how to actually implement these principles. This repository provides practical techniques for reducing the latency of GenAI applications.
☆37May 6, 2024Updated 2 years ago
Alternatives and similar repositories for The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications
Users that are interested in The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Real-time agent assist for contact centers, built on Azure Communication Services☆14Apr 1, 2025Updated last year
- This hands-on walks you through fine-tuning an open source LLM on Azure and serving the fine-tuned model on Azure. It is intended for Dat…☆12Jun 23, 2024Updated 2 years ago
- It summerizes the algorithms of Machine Learning.☆12Oct 26, 2025Updated 10 months ago
- Government of Indonesia's online datasets and information products.☆11May 28, 2014Updated 12 years ago
- A geometrical compiler project written in JavaScript for compiling and making complicated geometries.☆15Aug 28, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- State‑of‑the‑art speech recognition model for English, delivering transcription accuracy across diverse audio scenarios. <metadata> gpu: …☆20Apr 16, 2025Updated last year
- This solution demonstrates the setup and deployment of Amazon SageMaker Studio into a private VPC and implementation of multi-layer secur…☆22Jul 31, 2022Updated 4 years ago
- ☆15Apr 8, 2026Updated 4 months ago
- A collection of Korean NLP hands-on labs on Amazon SageMaker☆19Dec 20, 2023Updated 2 years ago
- ☆14Oct 30, 2021Updated 4 years ago
- Generative AI Ops RAG project template☆46Aug 20, 2026Updated 2 weeks ago
- A cloudflare worker that implements the vless server☆17Jan 22, 2025Updated last year
- Machine Learning Projects for the Dr. Andrew Ng's course on Coursera☆12Dec 19, 2017Updated 8 years ago
- Data from "Crowdsourcing of Parallel Corpora: the Case of Style Transfer for Detoxification" paper☆14Apr 3, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Coffee Chat Voice Assistant is a voice-driven ordering system powered by Azure OpenAI GPT-4o Realtime API, simulating the experience of o…☆32Jun 22, 2026Updated 2 months ago
- This project is uploaded as a demo to showcase various components and patterns☆11Oct 23, 2017Updated 8 years ago
- ☆10Dec 27, 2024Updated last year
- ☆12Dec 19, 2023Updated 2 years ago
- Courses and projects on Data Camp☆11Jun 28, 2020Updated 6 years ago
- Azure OpenAI benchmarking tool☆29Apr 4, 2025Updated last year
- ☆23Dec 7, 2023Updated 2 years ago
- Web Recipe, with React web app and Nestjs BFF☆10Sep 8, 2020Updated 5 years ago
- Kubernetes application log collector for Parseable server☆16Dec 2, 2022Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Experimental media project.☆15Jun 23, 2024Updated 2 years ago
- The Atlas of Pidgin and Creole Language Structures☆16Jul 29, 2026Updated last month
- 🚀 Embark on your agentic journey !☆29May 28, 2025Updated last year
- Simple apps that showing Movie data from MovieDB. Write using Kotlin with MVP patterns, RxJava, Dagger, and Retrofit.☆16Oct 29, 2019Updated 6 years ago
- R package that helps to render interlinear glossed linguistic examples in html rmarkdown documents and then semi-automatically compiles t…☆17Nov 18, 2025Updated 9 months ago
- XeLaTeX で和文する実験☆14Nov 6, 2022Updated 3 years ago
- KEDA External Scaler for Azure Cosmos DB☆13Updated this week
- [NeurIPS 2025] Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models☆19Apr 1, 2026Updated 5 months ago
- LaTeX macros to fill in PDF.☆13Jan 31, 2019Updated 7 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A Durable Task Python SDK compatible with the Durable Task Scheduler☆40Updated this week
- The Service Fabric Application Disaster Recovery Tool is a disaster recovery tool for Service Fabric clusters which allows users to recov…☆14Dec 8, 2022Updated 3 years ago
- Application of Blockchain in Crop Farming and Crop Supply☆10May 15, 2018Updated 8 years ago
- Official repo for the NCR Crypto Meetup☆17Jun 1, 2022Updated 4 years ago
- Repository that contains all the data, code and manuscript files for a paper that explores the use of GAMs in biomedical research.☆22Jun 3, 2024Updated 2 years ago
- Azure OpenAI benchmarking tool☆153May 28, 2024Updated 2 years ago
- Explore the use of DSPy for extracting features from PDFs 🔎☆54Mar 1, 2024Updated 2 years ago