nath1295 / MLX-Textgen
A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.
☆52Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for MLX-Textgen
- ☆38Updated 8 months ago
- Routing on Random Forest (RoRF)☆83Updated last month
- Scripts to create your own moe models using mlx☆86Updated 8 months ago
- Very basic framework for parameterized large language model (Q)LoRa fine-tuning using mlx, mlx_lm, and OgbujiPT. Architecture for system…☆34Updated this week
- ☆104Updated 7 months ago
- MLX-Embeddings is the best package for running Vision and Language Embedding models locally on your Mac using MLX.☆75Updated 3 weeks ago
- Distributed Inference for mlx LLm☆69Updated 3 months ago
- Fast parallel LLM inference for MLX☆145Updated 4 months ago
- Official homepage for "Self-Harmonized Chain of Thought"☆83Updated last month
- For inferring and serving local LLMs using the MLX framework☆89Updated 7 months ago
- Simple examples using Argilla tools to build AI☆38Updated last week
- ☆64Updated 5 months ago
- ☆110Updated 2 weeks ago
- tiny_fnc_engine is a minimal python library that provides a flexible engine for calling functions extracted from a LLM.☆37Updated 2 months ago
- run embeddings in MLX☆72Updated last month
- An easy-to-understand framework for LLM samplers that rewind and revise generated tokens☆105Updated 2 weeks ago
- A framework for evaluating function calls made by LLMs☆34Updated 3 months ago
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆110Updated 5 months ago
- Grammar checker with a keyboard shortcut for Ollama and Apple MLX with Automator on macOS.☆76Updated 9 months ago
- Large Model Proxy is designed to make it easy to run multiple resource-heavy Large Models (LM) on the same machine with limited amount of…☆46Updated last month
- run ollama & gguf easily with a single command☆47Updated 5 months ago
- Fast approximate inference on a single GPU with sparsity aware offloading☆38Updated 10 months ago
- Leveraging DSPy for AI-driven task understanding and solution generation, the Self-Discover Framework automates problem-solving through r…☆56Updated 3 months ago
- 🤖 Headless IDE for AI agents☆129Updated this week
- Gradio based tool to run opensource LLM models directly from Huggingface☆87Updated 4 months ago
- MLX port for xjdr's entropix sampler (mimics jax implementation)☆55Updated last week
- All the world is a play, we are but actors in it.☆47Updated 4 months ago
- An example implementation of RLHF (or, more accurately, RLAIF) built on MLX and HuggingFace.☆21Updated 4 months ago
- ☆117Updated this week