Alternatives and similar repositories for llm-hosting-container:

Users that are interested in llm-hosting-container are comparing it to the libraries listed below

• huggingface / optimum-neuron
  Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.
  ☆226Updated this week
• aws-neuron / aws-neuron-samples
  Example code for AWS Neuron SDK developers building inference and training applications
  ☆140Updated last week
• awslabs / extending-the-context-length-of-open-source-llms
  ☆53Updated 4 months ago
• aws-neuron / transformers-neuronx
  ☆104Updated 3 months ago
• aws-samples / fsi-genai-bootcamp
  ☆36Updated this week
• aws-neuron / upstreaming-to-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆13Updated last week
• aws-samples / llm-apps-workshop
  Use LLMs for building real-world apps
  ☆110Updated 3 months ago
• cohere-ai / cohere-aws
  ☆61Updated 2 weeks ago
• arunprsh / knowledge-augmented-LLMs
  Context is Key: Combining Embedding-based Retrieval with LLMs for Comprehensive Knowledge Enrichment
  ☆32Updated last year
• aws-samples / sagemaker-vector-store-microservice
  ☆22Updated last year
• aws-samples / llm-evaluation-methodology
  ☆42Updated 5 months ago
• aws-samples / sagemaker-distributed-training-workshop
  Hands-on workshop for distributed training and hosting on SageMaker
  ☆134Updated 2 weeks ago
• aws-samples / amazon-sagemaker-managed-spot-training
  Amazon SageMaker Managed Spot Training Examples
  ☆51Updated 9 months ago
• aws / fmeval
  Foundation Model Evaluations Library
  ☆243Updated last week
• awslabs / agent-evaluation
  A generative AI-powered framework for testing virtual agents.
  ☆216Updated 2 weeks ago
• wenqiglantz / edd-recursive-doc-agent-vs-metadata-replacement
  Applying Evaluation Driven Development (EDD) to aid in the design decision of RAG pipelines
  ☆31Updated last year
• aws / sagemaker-huggingface-inference-toolkit
  ☆253Updated 2 weeks ago
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated 10 months ago
• aws / sagemaker-pytorch-inference-toolkit
  Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at h…
  ☆138Updated 6 months ago
• aws-samples / foundation-model-benchmarking-tool
  Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stac…
  ☆238Updated this week
• aws-samples / whisper-audio-inference-for-amazon-sagemaker
  This is a short example showing how to utilize Amazon SageMaker's real time endpoints with OpenAI's open source Whisper model for audio t…
  ☆67Updated last year
• aws-samples / amazon-sagemaker-generativeai
  Repository for training and deploying Generative AI models, including text-text, text-to-image generation and prompt engineering playgrou…
  ☆143Updated this week
• aws-neuron / aws-neuron-parallelcluster-samples
  ☆24Updated last year
• philschmid / sagemaker-huggingface-llama-2-samples
  ☆88Updated last year
• JGalego / RAGmap
  A simple Streamlit application to visualize document chunks and queries in embedding space 🗺️🔍
  ☆13Updated this week
• philschmid / cdk-samples
  ☆57Updated 2 years ago
• aws-samples / rag-with-amazon-opensearch-and-sagemaker
  Question Answering Generative AI application with Large Language Models (LLMs) and Amazon OpenSearch Service
  ☆24Updated 4 months ago
• aws-neuron / neuronx-distributed
  ☆54Updated last week
• aws-samples / awsome-inference
  ☆45Updated 2 months ago
• aws-samples / rds-postgresql-pgvector
  ☆36Updated last year