EmbeddedLLM / embeddedllm
EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU
☆37Updated 6 months ago
Alternatives and similar repositories for embeddedllm:
Users that are interested in embeddedllm are comparing it to the libraries listed below
- Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)☆76Updated 2 months ago
- LLM reads a paper and produce a working prototype☆52Updated 2 weeks ago
- Serving CrewAI Agent as REST API with BentoML, optionally with self-host open-source LLMs☆16Updated 4 months ago
- ☆58Updated last month
- Transform unstructured documents into actionable, structured data with enterprise-grade precision and reliability, ready for large-scale …☆19Updated last week
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆67Updated 5 months ago
- Automatic Prompt Optimization☆34Updated 11 months ago
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆78Updated last month
- Examples of RAG using Llamaindex with local LLMs - Gemma, Mixtral 8x7B, Llama 2, Mistral 7B, Orca 2, Phi-2, Neural 7B☆126Updated last year
- Code for the EMNLP'24 paper "Learning to Extract Structured Entities Using Language Models"☆34Updated 3 weeks ago
- ☆41Updated 4 months ago
- ☆11Updated 10 months ago
- Self-host LLMs with vLLM and BentoML☆106Updated last week
- 🔎 A deep-dive into HyDE for Advanced LLM RAG + 💡 Introducing AutoHyDE, a semi-supervised framework to improve the effectiveness, covera…☆32Updated last year
- Streamlit Web UI for AGiXT☆26Updated 2 months ago
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆58Updated 3 weeks ago
- ☆21Updated 5 months ago
- ☆46Updated last year
- OpenMindedChatbot is a Proof Of Concept that leverages the power of Open source Large Language Models (LLM) with Function Calling capabil…☆29Updated last year
- LangChain + LiteLLM that works☆39Updated last week
- Leveraging DSPy for AI-driven task understanding and solution generation, the Self-Discover Framework automates problem-solving through r…☆60Updated 9 months ago
- Example implementation of Iteration of Tought - Gives a star if you like the project☆40Updated 4 months ago
- ☆38Updated this week
- ☆57Updated 2 months ago
- GPU prices aggregator for cloud providers☆36Updated this week
- ☆74Updated 3 months ago
- Own your AI, search the web with it🌐😎☆84Updated 3 months ago
- ☆44Updated 9 months ago
- ☆38Updated 2 weeks ago
- Uses a Gradio interface to stream coding related responses from local and cloud based large language models. Pulls context from GitHub Re…☆21Updated last month