WebGPU LLM inference tuned by hand
☆153Jun 24, 2023Updated 3 years ago
Alternatives and similar repositories for token-hawk
Users that are interested in token-hawk are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A guidance language for controlling large language models.☆40Jun 9, 2023Updated 3 years ago
- Inference Llama 2 in one file of pure JavaScript(HTML)☆35May 20, 2025Updated last year
- Deploy your GGML models to HuggingFace Spaces with Docker and gradio☆37Jun 6, 2023Updated 3 years ago
- Official code for ACL 2023 (short, findings) paper "Recursion of Thought: A Divide and Conquer Approach to Multi-Context Reasoning with L…☆45Jun 13, 2023Updated 3 years ago
- GBNF syntax high-lighter for Visual Studio Code☆15Dec 18, 2025Updated 10 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependenci…☆314Jan 31, 2024Updated 2 years ago
- Falcon LLM ggml framework with CPU and GPU support☆251Jul 2, 2026Updated 4 months ago
- Suno AI's Bark model in C/C++ for fast text-to-speech generation☆869Nov 16, 2024Updated last year
- Graphical user inferface for local language models☆10May 24, 2024Updated 2 years ago
- Erudito: Easy API/CLI to ask questions about your documentation☆97Nov 6, 2023Updated 2 years ago
- Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)☆572Aug 8, 2023Updated 3 years ago
- C++ implementation for 💫StarCoder☆459Sep 9, 2023Updated 3 years ago
- LLM-based code completion engine☆192Jan 23, 2025Updated last year
- ggml implementation of BERT☆506Feb 23, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Simple Tool Caller for llama.cpp☆11Aug 12, 2024Updated 2 years ago
- Browser-based Voice Assistant☆40Mar 31, 2023Updated 3 years ago
- ☆11Jun 27, 2025Updated last year
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust☆42Aug 2, 2023Updated 3 years ago
- Mixture of Expert (MoE) techniques for enhancing LLM performance through expert-driven prompt mapping and adapter combinations.☆11Feb 11, 2024Updated 2 years ago
- A dynamic compression method that selects consecutive layers from top to bottom using a pre-defined similarity threshold, and compacts pa…☆12Dec 4, 2025Updated 10 months ago
- ☆73Mar 15, 2024Updated 2 years ago
- Python examples using the bigcode/tiny_starcoder_py 159M model to generate code☆44May 31, 2023Updated 3 years ago
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,886Jan 28, 2024Updated 2 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Train llama with lora on one 4090 and merge weight of lora to work as stanford alpaca.☆53Jun 16, 2023Updated 3 years ago
- Efficient 3bit/4bit quantization of LLaMA models☆18May 18, 2023Updated 3 years ago
- Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…☆142Oct 17, 2023Updated 3 years ago
- ☆453Oct 15, 2023Updated 3 years ago
- Downsampling array of intervals☆26Dec 11, 2019Updated 6 years ago
- Fast Clojure interpreter and template engine☆14Oct 15, 2021Updated 5 years ago
- Natural Language Processing Plugin for Unreal Engine 4 using Tensorflow☆10Jun 16, 2019Updated 7 years ago
- various experiments for scaling inference time compute with small reasoning models☆17Jan 16, 2025Updated last year
- INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model☆1,590Mar 23, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- An MCP server that provides task-based workflow management for agents☆11Feb 28, 2026Updated 8 months ago
- Scripts to create your own moe models using mlx☆93Feb 26, 2024Updated 2 years ago
- Client-side faceted search engine☆29Sep 17, 2024Updated 2 years ago
- A Next.js chatbot app demonstrating seamless integration with window.ai.☆16Jun 25, 2023Updated 3 years ago
- This is a simple demonstration app for fast.ai using hugging face api's.☆10Oct 5, 2022Updated 4 years ago
- Automated testing of browser rendering engines using Clojure, generative (property-based) testing, formal grammars (EBNF), and a consensu…☆13Nov 18, 2019Updated 6 years ago
- A fork of textgen that kept some things like Exllama and old GPTQ.☆22Aug 20, 2024Updated 2 years ago