EmbeddedLLM / embeddedllmLinks
EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU
☆50Updated last year
Alternatives and similar repositories for embeddedllm
Users that are interested in embeddedllm are comparing it to the libraries listed below
Sorting:
- Self-host LLMs with vLLM and BentoML☆167Updated last week
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆114Updated 9 months ago
- Example implementation of Iteration of Tought - Gives a star if you like the project☆41Updated last year
- Own your AI, search the web with it🌐😎☆94Updated last year
- Elasticsearch integration into LangChain☆72Updated last week
- ⚡ Bhumi – The fastest AI inference client for Python, built with Rust for unmatched speed, efficiency, and scalability 🚀☆63Updated last week
- Rank LLMs, RAG systems, and prompts using automated head-to-head evaluation☆108Updated last year
- Simple examples using Argilla tools to build AI☆57Updated last year
- Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)☆92Updated last year
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆83Updated last year
- ☆57Updated last week
- ☆43Updated 2 months ago
- Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)☆82Updated 11 months ago
- Data generation and training repository for SERA: Soft-Verified Efficient Repository Agents.☆76Updated this week
- Benchmark and optimize LLM inference across frameworks with ease☆158Updated 4 months ago
- ☆24Updated 11 months ago
- ☆80Updated 4 months ago
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆90Updated last month
- ☆39Updated last year
- LLM reads a paper and produce a working prototype☆60Updated 9 months ago
- GPT-4 Level Conversational QA Trained In a Few Hours☆65Updated last year
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆59Updated 3 weeks ago
- ☆57Updated 11 months ago
- 🔎 A deep-dive into HyDE for Advanced LLM RAG + 💡 Introducing AutoHyDE, a semi-supervised framework to improve the effectiveness, covera…☆34Updated last year
- Automatic Prompt Optimization☆48Updated last year
- Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-…☆132Updated 11 months ago
- Route LLM requests to the best model for the task at hand.☆171Updated 2 weeks ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆90Updated 3 weeks ago
- 🚀 Scale your RAG pipeline using Ragswift: A scalable centralized embeddings management platform☆38Updated 2 years ago
- Nexusflow function call, tool use, and agent benchmarks.☆30Updated last year