EmbeddedLLM / embeddedllm
EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU
☆35Updated 5 months ago
Alternatives and similar repositories for embeddedllm:
Users that are interested in embeddedllm are comparing it to the libraries listed below
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆87Updated this week
- ☆41Updated 3 months ago
- Experiments with open source LLMs☆72Updated 2 weeks ago
- Own your AI, search the web with it🌐😎☆83Updated 2 months ago
- ☆20Updated last year
- LLM reads a paper and produce a working prototype☆51Updated 2 weeks ago
- Unsloth Studio☆74Updated 3 weeks ago
- Machine Learning Serving focused on GenAI with simplicity as the top priority.☆58Updated 2 months ago
- Ready-to-go containerized RAG service. Implemented with text-embedding-inference + Qdrant/LanceDB.☆63Updated 3 months ago
- RAGLight is a lightweight and modular Python library for implementing Retrieval-Augmented Generation (RAG), Agentic RAG and RAT (Retrieva…☆24Updated last week
- 👩🤝🤖 A curated list of datasets for large language models (LLMs), RLHF and related resources (continually updated)☆23Updated last year
- Very minimal (and stateless) agent framework☆41Updated 2 months ago
- ☆56Updated this week
- The official evaluation suite and dynamic data release for MixEval.☆11Updated 6 months ago
- Using LlamaIndex with Ray for productionizing LLM applications☆71Updated last year
- A list of language models with permissive licenses such as MIT or Apache 2.0☆24Updated last month
- Python Server for C3 AI app. A project that brings the power of Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) with…☆23Updated last year
- Deep research agent to help you find the best GitHub repositories 🕵️!☆54Updated this week
- Chat Complex PDF with Tables Using IBM WatsonX, Langchain and LlamaParser.☆12Updated 11 months ago
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆74Updated 2 weeks ago
- Official Repo for The Paper "Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systems"☆47Updated last month
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆64Updated 5 months ago
- ☆36Updated last month
- Code for ScribeAgent paper☆54Updated 3 weeks ago
- Uses a Gradio interface to stream coding related responses from local and cloud based large language models. Pulls context from GitHub Re…☆20Updated 3 weeks ago
- A project that brings the power of Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) within reach of everyone, particu…☆34Updated last year
- Build your own RAG and run it locally on your laptop: ColBERT + DSPy + Streamlit☆56Updated last year
- Command line tool for Deep Infra cloud ML inference service☆29Updated 9 months ago
- A framework for evaluating function calls made by LLMs☆37Updated 8 months ago
- ☆21Updated 4 months ago