ialacol / text-inference-batcher
A high performance batching router optimises max throughput for text inference workload
☆16Updated last year
Alternatives and similar repositories for text-inference-batcher:
Users that are interested in text-inference-batcher are comparing it to the libraries listed below
- Using modal.com to process FineWeb-edu data☆20Updated last month
- Radiantloom Email Assist 7B is an email-assistant large language model fine-tuned from Zephyr-7B-Beta, over a custom-curated dataset of 1…☆14Updated last year
- auto fine tune of models with synthetic data☆75Updated last year
- A guidance compatibility layer for llama-cpp-python☆34Updated last year
- ☆17Updated 4 months ago
- ☆16Updated 11 months ago
- Generate visual podcasts about novels using open source models☆25Updated 2 years ago
- Complex RAG backend☆28Updated last year
- OpenAI GPT hosted Agent Framework for Windows and MacOS☆36Updated 9 months ago
- A Python library to orchestrate LLMs in a neural network-inspired structure☆47Updated 7 months ago
- Very basic framework for composable parameterized large language model (Q)LoRA / (Q)Dora fine-tuning using mlx, mlx_lm, and OgbujiPT.☆40Updated 2 months ago
- ☆47Updated last year
- A daemon that makes a desktop OS accessible to AI agents☆26Updated 3 weeks ago
- ☆20Updated last year
- Embed anything.☆29Updated 11 months ago
- An intelligent code optimization system leveraging AI analysis, automated refactoring, and test generation. Built with DSPy and Gradio, i…☆18Updated 3 months ago
- the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly☆31Updated 6 months ago
- Zero-trust AI APIs for easy and private consumption of open-source LLMs☆40Updated 9 months ago
- ☆29Updated 5 months ago
- ☆22Updated last year
- ☆23Updated 5 months ago
- Conduct consumer interviews with synthetic focus groups using LLMs and LangChain☆43Updated last year
- 🦾💻🌐 distributed training & serverless inference at scale on RunPod☆17Updated 11 months ago
- ☆32Updated last year
- Data Questionnaire Agent Chatbot☆65Updated last week
- ☆41Updated last year
- ☆14Updated last year
- PyGPTPrompt: A CLI tool that manages context windows for AI models, facilitating user interaction and data ingestion for optimized long-t…☆29Updated 11 months ago
- A couple scripts to grab stats from email☆42Updated 7 months ago
- Code interpreter support for o1☆32Updated 7 months ago