awslabs / llm-hosting-containerLinks
Large Language Model Hosting Container
☆90Updated last week
Alternatives and similar repositories for llm-hosting-container
Users that are interested in llm-hosting-container are comparing it to the libraries listed below
Sorting:
- Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.☆242Updated this week
- ☆55Updated 3 months ago
- ☆62Updated 5 months ago
- TitanML Takeoff Server is an optimization, compression and deployment platform that makes state of the art machine learning models access…☆114Updated last year
- Applying Evaluation Driven Development (EDD) to aid in the design decision of RAG pipelines☆31Updated last year
- Streamlit app for recommending eval functions using prompt diffs☆29Updated last year
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆59Updated 2 months ago
- Fine-tune an LLM to perform batch inference and online serving.☆112Updated 4 months ago
- Leverage your LangChain trace data for fine tuning☆46Updated last year
- Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)☆79Updated 7 months ago
- ☆64Updated 6 months ago
- The backend behind the LLM-Perf Leaderboard☆10Updated last year
- Example code for AWS Neuron SDK developers building inference and training applications☆149Updated this week
- ☆111Updated 8 months ago
- A simple Streamlit application to visualize document chunks and queries in embedding space 🗺️🔍☆13Updated 5 months ago
- ☆87Updated 2 years ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated 2 years ago
- ☆199Updated last year
- experiments with inference on llama☆104Updated last year
- Tutorial to get started with SkyPilot!☆58Updated last year
- Amazon SageMaker Managed Spot Training Examples☆50Updated last year
- ☆29Updated 2 years ago
- Context is Key: Combining Embedding-based Retrieval with LLMs for Comprehensive Knowledge Enrichment☆32Updated 2 years ago
- ☆80Updated last year
- Recipes and resources for building, deploying, and fine-tuning generative AI with Fireworks.☆123Updated last week
- 💙 Unstructured Data Connectors for Haystack 2.0☆17Updated 2 years ago
- Deploy llama.cpp compatible Generative AI LLMs on AWS Lambda!☆173Updated last year
- ☆45Updated 11 months ago
- Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…☆49Updated last year
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆138Updated last year