ialacol / text-inference-batcher
A high performance batching router optimises max throughput for text inference workload
☆16Updated last year
Related projects ⓘ
Alternatives and complementary repositories for text-inference-batcher
- Using modal.com to process FineWeb-edu data☆19Updated 2 months ago
- ☆46Updated 7 months ago
- auto fine tune of models with synthetic data☆72Updated 9 months ago
- Simple examples using Argilla tools to build AI☆40Updated this week
- ☆104Updated 8 months ago
- Very basic framework for parameterized large language model (Q)LoRa fine-tuning using mlx, mlx_lm, and OgbujiPT. Architecture for system…☆35Updated last week
- Routing on Random Forest (RoRF)☆84Updated last month
- A guidance compatibility layer for llama-cpp-python☆34Updated last year
- GRDN.AI app for garden optimization☆69Updated 9 months ago
- ☆23Updated this week
- Distributed Inference for mlx LLm☆70Updated 3 months ago
- A python command-line tool to download & manage MLX AI models from Hugging Face.☆16Updated 2 months ago
- Embed anything.☆29Updated 5 months ago
- A Python library to orchestrate LLMs in a neural network-inspired structure☆41Updated last month
- A seamless matchmaking application that is programmed with Cohere Command R+, Stanford NLP DSPy framework, Weaviate Vector store and Crew…☆58Updated 7 months ago
- Radiantloom Email Assist 7B is an email-assistant large language model fine-tuned from Zephyr-7B-Beta, over a custom-curated dataset of 1…☆14Updated 10 months ago
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆53Updated 3 weeks ago
- Don't bug your friends with articles they'll never read. AI's have infinite attention, leverage them instead! Use the curation buddy to e…☆22Updated 6 months ago
- Ultra Fast Multi-Modality Vector Database☆17Updated 9 months ago
- Verbosity control for AI agents☆59Updated 5 months ago
- GPT-4 Level Conversational QA Trained In a Few Hours☆55Updated 3 months ago
- 🔓 The open-source autonomous agent LLM initiative 🔓☆90Updated 9 months ago
- Scripts to create your own moe models using mlx☆86Updated 8 months ago
- ☆30Updated last year
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆55Updated last week
- Demo of AI chatbot that predicts user message to generate response quickly.☆99Updated 8 months ago
- Code interpreter support for o1☆31Updated 2 months ago