distantmagic / paddler
Stateful load balancer custom-tailored for llama.cpp
☆563Updated this week
Related projects ⓘ
Alternatives and complementary repositories for paddler
- Replace OpenAI with Llama.cpp Automagically.☆289Updated 5 months ago
- Radient turns many data types (not just text) into vectors for similarity search, RAG, regression analysis, and more.☆268Updated 2 months ago
- Felafax is building AI infra for non-NVIDIA GPUs☆509Updated this week
- LLM Analytics☆615Updated last month
- An application for running LLMs locally on your device, with your documents, facilitating detailed citations in generated responses.☆498Updated 3 weeks ago
- Action library for AI Agent☆191Updated 2 weeks ago
- The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM …☆493Updated 3 months ago
- Multi-node production GenAI stack. Run the best of open source AI easily on your own servers. Easily add knowledge from documents and scr…☆348Updated last week
- Finetune llama2-70b and codellama on MacBook Air without quantization☆447Updated 7 months ago
- ai for jq☆234Updated 2 months ago
- Visualize the intermediate output of Mistral 7B☆313Updated 9 months ago
- Tech Stack for Building, Evaluating, and Deploying your LLM Application☆328Updated this week
- ☆162Updated 5 months ago
- An implementation of bucketMul LLM inference☆214Updated 4 months ago
- ☆727Updated 7 months ago
- GGUF implementation in C as a library and a tools CLI program☆244Updated 4 months ago
- This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient …☆210Updated last month
- Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.☆374Updated 8 months ago
- Bayesian Optimization as a Coverage Tool for Evaluating LLMs. Accurate evaluation (benchmarking) that's 10 times faster with just a few l…☆271Updated this week
- ➖ Stripped down, stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features are compl…☆217Updated last week
- Local voice chatbot for engaging conversations, powered by Ollama, Hugging Face Transformers, and Coqui TTS Toolkit☆718Updated 3 months ago
- Go library for embedded vector search and semantic embeddings using llama.cpp☆358Updated 3 weeks ago
- Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.☆836Updated 10 months ago
- Open Source Application for Advanced LLM Engineering: interact, train, fine-tune, and evaluate large language models on your own computer…☆416Updated this week
- Fast, SQL powered, in-process vector search for any language with an SQLite driver☆268Updated 2 weeks ago
- MLX-VLM is a package for running Vision LLMs locally on your Mac using MLX.☆496Updated this week
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆221Updated 6 months ago
- WebAssembly binding for llama.cpp - Enabling in-browser LLM inference☆438Updated 2 weeks ago
- 🚀 Retrieval Augmented Generation (RAG) with txtai. Combine search and LLMs to find insights with your own data.☆281Updated this week
- A fast batching API to serve LLM models☆172Updated 6 months ago