Alternatives and similar repositories for mistral-tokenizer

Users that are interested in mistral-tokenizer are comparing it to the libraries listed below

• IntrinsicLabsAI / gbnfgen
  TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces
  ☆141Updated last year
• belladoreai / llama3-tokenizer-js
  JS tokenizer for LLaMA 3 and LLaMA 3.1
  ☆117Updated 4 months ago
• granawkins / latent-dictionary
  A dictionary, but it shows you position in embedding space relative to some synonyms/antonyms instead of a definition.
  ☆74Updated 2 weeks ago
• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆65Updated 2 years ago
• abacaj / openhermes-function-calling
  ☆135Updated 2 years ago
• ahoylabs / gguf.js
  A Javascript library (with Typescript types) to parse metadata of GGML based GGUF files.
  ☆51Updated last year
• tairov / QStarLearning.mojo
  ☆112Updated 2 years ago
• glaive-ai / function-calling-server
  ☆36Updated last year
• Technoculture / personal-graph
  Simple Graph Memory for AI applications
  ☆89Updated 6 months ago
• belladoreai / llama-tokenizer-js
  JS tokenizer for LLaMA 1 and 2
  ☆362Updated last year
• 4dh / GRDN
  GRDN.AI app for garden optimization
  ☆69Updated 2 weeks ago
• yoheinakajima / predictivechat
  Demo of AI chatbot that predicts user message to generate response quickly.
  ☆104Updated last year
• OoriData / Toolio
  GenAI & agent toolkit for Apple Silicon Mac, implementing JSON schema-steered structured output (3SO) and tool-calling in Python. For mor…
  ☆129Updated last month
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆119Updated last year
• mzbac / mlx-lora
  ☆38Updated last year
• cfahlgren1 / natural-functions
  Mistral-7B finetuned for function calling
  ☆16Updated last year
• IntrinsicLabsAI / grammar-builder
  Generates grammer files from typescript for LLM generation
  ☆38Updated last year
• mzbac / mlx_sharding
  Distributed Inference for mlx LLm
  ☆99Updated last year
• hrishioa / wasm-ai
  Vercel and web-llm template to run wasm models directly in the browser.
  ☆164Updated 2 years ago
• teknium1 / ShareGPT-Builder
  ☆117Updated 11 months ago
• taylorai / mlx_embedding_models
  run embeddings in MLX
  ☆96Updated last year
• markokraemer / OACE
  ☆22Updated last year
• Extensible-AI / DAGent
  Build AI Agents with Your Existing Python Code!
  ☆70Updated last year
• andreasjansson / AutoCog
  ☆40Updated 6 months ago
• j-csc / mlx_bark
  Port of Suno's Bark TTS transformer in Apple's MLX Framework
  ☆86Updated last year
• desik1998 / MathWithLLMs
  ☆50Updated last year
• whitphx / transformers.js.py
  ☆88Updated last month
• swyxio / openlangmem
  ☆47Updated last year
• LachlanGray / lmql-tree-of-thoughts
  LMQL implementation of tree of thoughts
  ☆34Updated last year
• SpellcraftAI / turing
  Turing machines, Rule 110, and A::B reversal using Claude 3 Opus.
  ☆58Updated last year