Alternatives and similar repositories for FastAPI-LLM-Model-Serving

Users that are interested in FastAPI-LLM-Model-Serving are comparing it to the libraries listed below

• anyscale / e2e-llm-workflows
  Fine-tune an LLM to perform batch inference and online serving.
  ☆120Updated 8 months ago
• decodingai-magazine / articles-code
  💻 Decoding ML articles hub: Hands-on articles with code on production-grade ML
  ☆141Updated 11 months ago
• aishwaryaprabhat / goku
  GenAIOps on Kubernetes: A collection of reference architectures for running GenAI at scale on Kubernetes using OSS tooling
  ☆135Updated last year
• huggingbench / huggingbench
  Find the optimal model serving solution for 🤗 Hugging Face models 🚀
  ☆45Updated 6 months ago
• zenml-io / zenml-projects
  A repository for all ZenML projects that are specific production use-cases.
  ☆302Updated 2 months ago
• deepset-ai / rag-with-nvidia-nims
  🚀 Use NVIDIA NIMs with Haystack pipelines
  ☆32Updated last year
• amogkam / llama_index_ray
  Using LlamaIndex with Ray for productionizing LLM applications
  ☆71Updated 2 years ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆168Updated 2 weeks ago
• deep-diver / llamaduo
  [ACL'25] Official Code for LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs
  ☆314Updated 6 months ago
• YeonwooSung / MLOps
  Miscellaneous codes and writings for MLOps
  ☆15Updated 3 weeks ago
• ibm-self-serve-assets / SuperKnowa
  Build Enterprise RAG (Retriver Augmented Generation) Pipelines to tackle various Generative AI use cases with LLM's by simply plugging co…
  ☆117Updated last year
• anyscale / multimodal-ai
  Multimodal AI workloads: batch inference, model training and online serving.
  ☆106Updated 5 months ago
• anakin87 / mistral-haystack
  Mistral + Haystack: build RAG pipelines that rock 🤘
  ☆106Updated 2 years ago
• jayita13 / GenerativeAI
  GenAI Experimentation
  ☆59Updated 5 months ago
• peremartra / optipfair
  Structured pruning and bias visualization for Large Language Models. Tools for LLM optimization and fairness analysis.
  ☆27Updated this week
• rajshah4 / LLM-Evaluation
  Sample notebooks and prompts for LLM evaluation
  ☆159Updated 3 months ago
• AI-Maker-Space / Fine-tuning-LLM-Resources
  A collection of fine-tuning notebooks!
  ☆30Updated 2 years ago
• Paulescu / virtual-assistant-llm
  ☆56Updated last year
• marklysze / LlamaIndex-RAG-WSL-CUDA
  Examples of RAG using Llamaindex with local LLMs - Gemma, Mixtral 8x7B, Llama 2, Mistral 7B, Orca 2, Phi-2, Neural 7B
  ☆130Updated last year
• run-llama / awesome-rag
  ☆75Updated last year
• dipanjanS / improving-RAG-systems-dhs2024
  This repository will contain the presentation and python jupyter notebooks for the DataHack Summit 2024 conference talk, Improving Real-w…
  ☆121Updated last year
• ThinamXx / Meta-llama
  Complete implementation of Llama2 with/without KV cache & inference 🚀
  ☆49Updated last year
• NVIDIA / workbench-example-mistral-finetune
  An NVIDIA AI Workbench example project for fine-tuning a Mistral 7B model
  ☆69Updated last year
• AntonioGr7 / pratical-llms
  A collection of hand on notebook for LLMs practitioner
  ☆51Updated last year
• langchain-ai / langchain-nvidia
  ☆184Updated last week
• PrithivirajDamodaran / Route0x
  Low latency, High Accuracy, Custom Query routers for Humans and Agents. Built by Prithivi Da
  ☆119Updated 10 months ago
• aniketmaurya / llm-inference
  Large Language Model (LLM) Inference API and Chatbot
  ☆128Updated last year
• ananthanarayanan431 / Langchain-Projects-LLM
  Various projects using Large Language Model (GPT & LLAMA) other open source model from HuggingFace and OpenAI. OpenAI API required for ru…
  ☆112Updated last month
• NVIDIA / context-aware-rag
  Context-Aware RAG library for Knowledge Graph ingestion and retrieval functions.
  ☆54Updated last week
• AI-Maker-Space / The-AI-Engineer-Challenge
  Building your first LLM application with OpenAI, and AI-assisted Development, step-by-step!
  ☆123Updated 2 months ago