ksm26 / Efficiently-Serving-LLMsLinks
Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.
☆17Updated last year
Alternatives and similar repositories for Efficiently-Serving-LLMs
Users that are interested in Efficiently-Serving-LLMs are comparing it to the libraries listed below
Sorting:
- Fine-tune an LLM to perform batch inference and online serving.☆115Updated 6 months ago
- A compute framework for building Search, RAG, Recommendations and Analytics over complex (structured+unstructured) data, with ultra-modal…☆12Updated last year
- A set of scripts and notebooks on LLM finetunning and dataset creation☆112Updated last year
- Material for the series of seminars on Large Language Models☆34Updated last year
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.☆32Updated 3 months ago
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆78Updated last year
- Model, Code & Data for the EMNLP'23 paper "Making Large Language Models Better Data Creators"☆137Updated 2 years ago
- Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…☆51Updated last year
- Code Repository for Blog - How to Productionize Large Language Models (LLMs)☆12Updated last year
- ☆80Updated last year
- Sample notebooks and prompts for LLM evaluation☆156Updated last month
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆49Updated last year
- Tuning the Finetuning: An exploration of achieving success with QLoRA☆45Updated last year
- Supervised instruction finetuning for LLM with HF trainer and Deepspeed☆36Updated 2 years ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆139Updated last year
- Seemless interface of using PyTOrch distributed with Jupyter notebooks☆57Updated 3 months ago
- This repository contains the implementation of evaluation metrics for recommendation systems. We have compared similarity, candidate gene…☆27Updated 10 months ago
- Distributed training (multi-node) of a Transformer model☆90Updated last year
- Examples of using Evidently to evaluate, test and monitor ML models.☆43Updated last week
- LLM_library is a comprehensive repository serves as a one-stop resource hands-on code, insightful summaries.☆69Updated last year
- Fine-tuning large language models (LLMs) is crucial for enhancing performance across domain-specific task applications. This comprehensiv…☆12Updated last year
- ☆15Updated 2 years ago
- ☆43Updated last year
- experiments with inference on llama☆103Updated last year
- Obsolete version of CUDA-mode repo -- use cuda-mode/lectures instead☆26Updated last year
- A RAG that can scale 🧑🏻💻☆11Updated last year
- Collection of links, tutorials and best practices of how to collect the data and build end-to-end RLHF system to finetune Generative AI m…☆224Updated 2 years ago
- ☆86Updated 2 years ago
- Build Enterprise RAG (Retriver Augmented Generation) Pipelines to tackle various Generative AI use cases with LLM's by simply plugging co…☆116Updated last year
- ☆89Updated 2 years ago