aws-samples / scalable-hw-agnostic-inference
A hardware-agnostic (NVIDIA's GPUs and AWS Inferentia accelerators) deployment of computer-vision models (e.g., YOLO, ViT), generate text and text-to-image (e.g., Llama3 and Stable Diffusion ) on EKS controlled by K8s ingress in routing-time and Karpenter in scheduling-time that is scaled by KEDA.
☆20Updated last week
Alternatives and similar repositories for scalable-hw-agnostic-inference:
Users that are interested in scalable-hw-agnostic-inference are comparing it to the libraries listed below
- ☆44Updated last month
- AIOps modules is a collection of reusable Infrastructure as Code (IaC) modules for Machine Learning (ML), Foundation Models (FM), Large L…☆71Updated last week
- This Guidance demonstrates how to deploy a machine learning inference architecture on Amazon Elastic Kubernetes Service (Amazon EKS). It …☆42Updated 2 months ago
- Create and manage Amazon SageMaker HyperPod clusters, run distributed model training☆19Updated this week
- ☆20Updated 3 weeks ago
- ☆60Updated last year
- CDK construct for installing and configuring Karpenter on EKS clusters☆42Updated last week
- ☆21Updated 2 months ago
- This repository aims to showcase how to finetune a FM model in Amazon EKS cluster using, JupyterHub to provision notebooks and craft both…☆44Updated 9 months ago
- ☆23Updated last month
- ☆11Updated 4 months ago
- ☆51Updated last week
- ☆9Updated 11 months ago
- 'Talk to your slide deck' (Multimodal RAG) using foundation models (FMs) hosted on Amazon Bedrock and Amazon SageMaker☆40Updated 2 months ago
- ☆32Updated 2 months ago
- FM-Leaderboard-er allows you to create leaderboard to find the best LLM/prompt for your own business use case based on your data, task, p…☆18Updated 5 months ago
- Create an Amazon EKS cluster and run a distributed training example☆28Updated 7 months ago
- Run FMBench simultaneously across multiple Amazon EC2 machines to benchmark an FM across multiple serving stacks simultaneously☆12Updated 3 weeks ago
- ☆23Updated last week
- Deploy and scale distributed python applications on Amazon EKS using Ray☆11Updated 3 weeks ago
- The repository includes integrations with Amazon Bedrock and its included LLM, such as Amazon Titan and vector and graph database for a R…☆8Updated 4 months ago
- ACK service controller for Amazon SageMaker☆43Updated this week
- Create, List, Update, Delete Amazon EKS clusters. Deploy and manage software on EKS. Run distributed model training and inference example…☆55Updated last week
- ☆37Updated 4 months ago
- aws-solutions-library-samples / guidance-for-a-multi-tenant-generative-ai-gateway-with-cost-and-usage-tracking-on-awsThis Guidance demonstrates how to build an internal Software-as-a-Service (SaaS) platform that provides access to foundation models, like…☆73Updated 5 months ago
- aws-solutions-library-samples / guidance-for-automated-provisioning-of-application-ready-amazon-eks-clustersThe EKS workload accelerator is a collection of reference implementations for Amazon EKS designed to accelerate the time it takes to prov…☆31Updated 2 weeks ago
- Custom kube-scheduler for binpacking targeting Spark on EKS and other jobs workloads☆17Updated last week
- This Guidance demonstrates how to create an intelligent manufacturing digital thread through a combination of knowledge graph and generat…☆21Updated 4 months ago
- ☆54Updated 2 months ago
- ☆20Updated 2 weeks ago