Alternatives and similar repositories for FastAPI-LLM-Model-Serving

Users that are interested in FastAPI-LLM-Model-Serving are comparing it to the libraries listed below

• anyscale / e2e-llm-workflows
  Fine-tune an LLM to perform batch inference and online serving.
  ☆115Updated 7 months ago
• huggingbench / huggingbench
  Find the optimal model serving solution for 🤗 Hugging Face models 🚀
  ☆45Updated 5 months ago
• deep-diver / llamaduo
  [ACL'25] Official Code for LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs
  ☆314Updated 5 months ago
• aishwaryaprabhat / goku
  GenAIOps on Kubernetes: A collection of reference architectures for running GenAI at scale on Kubernetes using OSS tooling
  ☆135Updated last year
• rajshah4 / LLM-Evaluation
  Sample notebooks and prompts for LLM evaluation
  ☆158Updated 2 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆163Updated last month
• decodingai-magazine / articles-code
  💻 Decoding ML articles hub: Hands-on articles with code on production-grade ML
  ☆140Updated 10 months ago
• anyscale / multimodal-ai
  Multimodal AI workloads: batch inference, model training and online serving.
  ☆105Updated 4 months ago
• jayita13 / GenerativeAI
  GenAI Experimentation
  ☆59Updated 4 months ago
• AntonioGr7 / pratical-llms
  A collection of hand on notebook for LLMs practitioner
  ☆51Updated 11 months ago
• deepset-ai / rag-with-nvidia-nims
  🚀 Use NVIDIA NIMs with Haystack pipelines
  ☆31Updated last year
• tcapelle / llm_recipes
  A set of scripts and notebooks on LLM finetunning and dataset creation
  ☆113Updated last year
• zenml-io / zenml-projects
  A repository for all ZenML projects that are specific production use-cases.
  ☆291Updated last month
• AI-Maker-Space / Fine-tuning-LLM-Resources
  A collection of fine-tuning notebooks!
  ☆29Updated 2 years ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆93Updated 10 months ago
• vincentclaes / classification-with-llm
  How far can we go with an LLM for a classification problem
  ☆24Updated last year
• amogkam / llama_index_ray
  Using LlamaIndex with Ray for productionizing LLM applications
  ☆71Updated 2 years ago
• dipanjanS / improving-RAG-systems-dhs2024
  This repository will contain the presentation and python jupyter notebooks for the DataHack Summit 2024 conference talk, Improving Real-w…
  ☆121Updated last year
• anyscale / ray-summit-2023-training
  ☆89Updated 2 years ago
• NVIDIA / metropolis-nim-workflows
  Collection of reference workflows for building intelligent agents with NIMs
  ☆183Updated 11 months ago
• wenqiglantz / hands-on-llamaindex
  A Hands-on Practical Guide to LlamaIndex
  ☆33Updated last year
• YeonwooSung / MLOps
  Miscellaneous codes and writings for MLOps
  ☆15Updated last month
• marklysze / LlamaIndex-RAG-WSL-CUDA
  Examples of RAG using Llamaindex with local LLMs - Gemma, Mixtral 8x7B, Llama 2, Mistral 7B, Orca 2, Phi-2, Neural 7B
  ☆131Updated last year
• ksm26 / Efficiently-Serving-LLMs
  Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and L…
  ☆17Updated last year
• deep-diver / gpt2-ft-pipeline
  GPT2 fine-tuning pipeline with KerasNLP, TensorFlow, and TensorFlow Extended
  ☆33Updated 2 years ago
• NVIDIA / context-aware-rag
  Context-Aware RAG library for Knowledge Graph ingestion and retrieval functions.
  ☆49Updated 2 months ago
• evidentlyai / community-examples
  Examples of using Evidently to evaluate, test and monitor ML models.
  ☆45Updated 3 weeks ago
• run-llama / awesome-rag
  ☆75Updated last year
• ThinamXx / Meta-llama
  Complete implementation of Llama2 with/without KV cache & inference 🚀
  ☆49Updated last year
• PrithivirajDamodaran / Route0x
  Low latency, High Accuracy, Custom Query routers for Humans and Agents. Built by Prithivi Da
  ☆119Updated 9 months ago