Alternatives and similar repositories for mistral-tokenizer

Users that are interested in mistral-tokenizer are comparing it to the libraries listed below

• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆66Updated last year
• FL33TW00D / laserbeak
  Add local LLMs to your Web or Electron apps! Powered by Rust + WebGPU
  ☆102Updated 2 years ago
• IntrinsicLabsAI / gbnfgen
  TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces
  ☆137Updated 11 months ago
• OoriData / Toolio
  GenAI & agent toolkit for Apple Silicon Mac, implementing JSON schema-steered structured output (3SO) and tool-calling in Python. For mor…
  ☆126Updated last month
• ahoylabs / gguf.js
  A Javascript library (with Typescript types) to parse metadata of GGML based GGUF files.
  ☆47Updated 10 months ago
• abacaj / openhermes-function-calling
  ☆135Updated last year
• 4dh / GRDN
  GRDN.AI app for garden optimization
  ☆70Updated last year
• huggingface / ember
  ANE accelerated embedding models!
  ☆18Updated 6 months ago
• nicholasyager / llama-cpp-guidance
  A guidance compatibility layer for llama-cpp-python
  ☆35Updated last year
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆119Updated last year
• swyxio / simple-realtime-console
  OpenAI's Realtime API minus the enterprise bloat
  ☆46Updated 7 months ago
• Prismadic / magnet
  the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly
  ☆31Updated 8 months ago
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆75Updated last year
• teknium1 / ShareGPT-Builder
  ☆114Updated 6 months ago
• andreasjansson / AutoCog
  ☆40Updated last month
• yoheinakajima / predictivechat
  Demo of AI chatbot that predicts user message to generate response quickly.
  ☆103Updated last year
• granawkins / latent-dictionary
  A dictionary, but it shows you position in embedding space relative to some synonyms/antonyms instead of a definition.
  ☆74Updated 4 months ago
• Birch-san / mpt-play
  Command-line script for inferencing from models such as MPT-7B-Chat
  ☆101Updated last year
• swyxio / openlangmem
  ☆47Updated last year
• LachlanGray / lmql-tree-of-thoughts
  LMQL implementation of tree of thoughts
  ☆34Updated last year
• Alignment-Lab-AI / Our-Projects
  A repository of projects and datasets under active development by Alignment Lab AI
  ☆22Updated last year
• strangeloopcanon / Autotune_GPT
  Recursive self-improvement
  ☆55Updated last year
• dzhng / llamaflow
  The Typescript-first prompt engineering toolkit for working with chat based LLMs.
  ☆109Updated last year
• 567-labs / fastllm
  A collection of LLM services you can self host via docker or modal labs to support your applications development
  ☆191Updated last year
• Goekdeniz-Guelmez / mlx-kan
  KAN (Kolmogorov–Arnold Networks) in the MLX framework for Apple Silicon
  ☆18Updated last week
• ggerganov / seamless_communication
  Foundational Models for State-of-the-Art Speech and Text Translation
  ☆31Updated last year
• mzbac / mlx-lora
  ☆38Updated last year
• smolorg / smoltropix
  MLX port for xjdr's entropix sampler (mimics jax implementation)
  ☆64Updated 7 months ago
• SpellcraftAI / turing
  Turing machines, Rule 110, and A::B reversal using Claude 3 Opus.
  ☆59Updated last year
• otriscon / llm-structured-output
  ☆87Updated 5 months ago