deepsense-ai / edge-slmLinks
This project is a native implementation of a RAG pipeline for Small Language Models tested on Android devices. The main goal was to fit the whole RAG pipeline into a resource constrained device - ie. smartphone. By design the provided RAG library should be deployable on various platforms.
☆93Updated last year
Alternatives and similar repositories for edge-slm
Users that are interested in edge-slm are comparing it to the libraries listed below
Sorting:
- Self-host LLMs with vLLM and BentoML☆150Updated last week
- Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector…☆320Updated 11 months ago
- ☆207Updated last year
- One click templates for inferencing Language Models☆213Updated 2 months ago
- ☆35Updated 8 months ago
- LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's architecture in a simpler manner.☆186Updated last year
- Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…☆179Updated last year
- Inference, Fine Tuning and many more recipes with Gemma family of models☆269Updated 2 months ago
- A Demo of Cache-Augmented Generation (CAG) in an LLM☆109Updated 3 months ago
- ☆388Updated this week
- Awesome Mobile LLMs☆246Updated last week
- Comparison of Language Model Inference Engines☆229Updated 9 months ago
- An innovative library for efficient LLM inference via low-bit quantization☆350Updated last year
- Build datasets using natural language☆529Updated 2 weeks ago
- A collection of all available inference solutions for the LLMs☆91Updated 7 months ago
- [ACL'25] Official Code for LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs☆314Updated 2 months ago
- ☆152Updated 3 weeks ago
- ☆237Updated 3 months ago
- Own your AI, search the web with it🌐😎☆90Updated 8 months ago
- FRP Fork☆175Updated 6 months ago
- Open Source Text Embedding Models with OpenAI Compatible API☆160Updated last year
- Recipes for learning, fine-tuning, and adapting ColPali to your multimodal RAG use cases. 👨🏻🍳☆336Updated 4 months ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆596Updated last week
- [EMNLP 2024 Demo] TinyAgent: Function Calling at the Edge!☆451Updated last year
- 🤗 Benchmark Large Language Models Reliably On Your Data☆398Updated this week
- An Open Source Toolkit For LLM Distillation☆732Updated 2 months ago
- Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.☆243Updated 2 months ago
- Speech To Speech: an effort for an open-sourced and modular GPT4-o☆69Updated 11 months ago
- A collection of the the best ML and AI news every week (research, news, resources)☆171Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆266Updated 11 months ago