ialacol / text-inference-batcher
A high performance batching router optimises max throughput for text inference workload
☆16Updated last year
Alternatives and similar repositories for text-inference-batcher:
Users that are interested in text-inference-batcher are comparing it to the libraries listed below
- Using modal.com to process FineWeb-edu data☆20Updated 2 months ago
- Very basic framework for parameterized large language model (Q)LoRA / (Q)Dora fine-tuning using mlx, mlx_lm, and OgbujiPT. Architecture …☆37Updated this week
- Radiantloom Email Assist 7B is an email-assistant large language model fine-tuned from Zephyr-7B-Beta, over a custom-curated dataset of 1…☆14Updated last year
- ☆22Updated last year
- A guidance compatibility layer for llama-cpp-python☆34Updated last year
- Complex RAG backend☆28Updated 10 months ago
- ☆16Updated 2 months ago
- The Swarm Ecosystem☆19Updated 6 months ago
- ☆31Updated last year
- Easily create LLM automation/agent workflows☆58Updated last year
- ☆19Updated 3 weeks ago
- ☆25Updated 2 months ago
- Leveraging DSPy for AI-driven task understanding and solution generation, the Self-Discover Framework automates problem-solving through r…☆58Updated 7 months ago
- Generate visual podcasts about novels using open source models☆25Updated 2 years ago
- 🚀 Scale your RAG pipeline using Ragswift: A scalable centralized embeddings management platform☆37Updated last year
- A daemon that makes a desktop OS accessible to AI agents☆20Updated this week
- GRDN.AI app for garden optimization☆70Updated last year
- Embed anything.☆29Updated 8 months ago
- ☆48Updated last year
- The Benefits of a Concise Chain of Thought on Problem Solving in Large Language Models☆21Updated 2 months ago
- fork of litellm that is open source☆15Updated 2 months ago
- A collection of example AI programs built using DSPy and maitained by the Langtrace AI team.☆21Updated 3 months ago
- Falcon40B and 7B (Instruct) with streaming, top-k, and beam search☆40Updated last year
- ☆16Updated 9 months ago
- ☆23Updated 3 months ago
- A Python library to orchestrate LLMs in a neural network-inspired structure☆46Updated 4 months ago
- ☆65Updated 8 months ago
- ☆111Updated 2 months ago
- Using langchain, deeplake and openai to create a Q&A on the Mojo lang programming manual☆22Updated last year
- ☆46Updated 10 months ago