alvarobartt / bpe.zig
Minimal implementation of a Byte Pair Encoding (BPE) tokenizer in Zig
☆11Updated 3 weeks ago
Alternatives and similar repositories for bpe.zig:
Users that are interested in bpe.zig are comparing it to the libraries listed below
- Tensor library for Zig☆11Updated 5 months ago
- A text embedding extension for the Polars Dataframe library.☆24Updated 5 months ago
- Thin wrapper around GGML to make life easier☆26Updated this week
- Rust crate for some audio utilities☆22Updated last month
- A high-performance constrained decoding engine based on context free grammar in Rust☆50Updated 3 months ago
- ☆19Updated 6 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- NLP with Rust for Python 🦀🐍☆62Updated 10 months ago
- ☆11Updated 3 months ago
- A small rust-based data loader☆24Updated 4 months ago
- Journeys between the two worlds of Python 🐍 and Rust 🦀☆40Updated last week
- A Learning Journey: Micrograd in Mojo 🔥☆61Updated 6 months ago
- Modular Rust transformer/LLM library using Candle☆36Updated 11 months ago
- 8-bit floating point types for Rust☆46Updated last month
- Graph model execution API for Candle☆13Updated 5 months ago
- GPT-2 inference engine written in Zig☆37Updated last year
- TensorRT-LLM server with Structured Outputs (JSON) built with Rust☆49Updated 3 weeks ago
- 👷 Build compute kernels☆35Updated this week
- Inference Llama 2 in one file of pure Zig☆182Updated 5 months ago
- GPU based FFT written in Rust and CubeCL☆21Updated last month
- ☆54Updated 10 months ago
- Proof of concept for running moshi/hibiki using webrtc☆18Updated 2 months ago
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust☆38Updated last year
- ☆59Updated 2 years ago
- Inference Llama/Llama2/Llama3 Modes in NumPy☆20Updated last year
- The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…☆131Updated 4 months ago
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆80Updated last month
- Manage ML configuration with pydantic☆12Updated 4 months ago
- TOPLOC: is a novel method for verifiable inference that enables users to verify that LLM providers are using the correct model configurat…☆21Updated 2 weeks ago
- Because it's there.☆16Updated 7 months ago