Alternatives and similar repositories for tokeni.zig:

Users that are interested in tokeni.zig are comparing it to the libraries listed below

• charlesfrye / cuda-substrings
  Because it's there.
  ☆15Updated 6 months ago
• kyutai-labs / kaudio
  Rust crate for some audio utilities
  ☆22Updated 2 weeks ago
• kyutai-labs / yomikomi
  A small rust-based data loader
  ☆24Updated 3 months ago
• argilla-io / distilabel-spin-dibt
  Repository containing the SPIN experiments on the DIBT 10k ranked prompts
  ☆24Updated last year
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆61Updated 9 months ago
• thedataquarry / rustinpieces
  Journeys between the two worlds of Python 🐍 and Rust 🦀
  ☆39Updated this week
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• hscspring / llama.np
  Inference Llama/Llama2/Llama3 Modes in NumPy
  ☆20Updated last year
• nisten / grokadamw
  new optimizer
  ☆19Updated 7 months ago
• Dan-wanna-M / kbnf
  A high-performance constrained decoding engine based on context free grammar in Rust
  ☆48Updated 2 months ago
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆75Updated last year
• Narsil / hf-chat
  ☆25Updated 3 months ago
• LaurentMazare / glim
  ☆19Updated 5 months ago
• mixedbread-ai / batched
  The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…
  ☆126Updated 3 months ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆25Updated this week
• raphaelsty / neural-tree
  Tree-based indexes for neural-search
  ☆29Updated last year
• ahmed-moubtahij / TokenHealer
  ☆22Updated 9 months ago
• EricLBuehler / zig_ml
  Tensor library for Zig
  ☆11Updated 4 months ago
• wdoppenberg / polars-candle
  A text embedding extension for the Polars Dataframe library.
  ☆24Updated 4 months ago
• oxidized-transformers / oxidized-transformers
  Modular Rust transformer/LLM library using Candle
  ☆36Updated 10 months ago
• enjalot / latent-sae
  Training code for Sparse Autoencoders on Embedding models
  ☆38Updated last month
• kyutai-labs / moshi-webrtc
  Proof of concept for running moshi/hibiki using webrtc
  ☆18Updated last month
• huggingface / kernel-builder
  👷 Build compute kernels
  ☆17Updated this week
• allenai / adapt-demos
  Lightweight tools for quick and easy LLM demo's
  ☆26Updated 6 months ago
• charlesfrye / minimodal
  A miniature version of Modal
  ☆20Updated 9 months ago
• guidance-ai / llgtrt
  TensorRT-LLM server with Structured Outputs (JSON) built with Rust
  ☆44Updated 3 weeks ago
• huggingface / leaderboards
  ☆15Updated 2 weeks ago
• cognitivecomputations / kraken
  ☆66Updated 10 months ago
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆34Updated 3 months ago
• gau-nernst / learn-cuda
  Learn CUDA with PyTorch
  ☆19Updated last month