Alternatives and similar repositories for mistral-tokenizer

Users that are interested in mistral-tokenizer are comparing it to the libraries listed below

• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆66Updated last year
• belladoreai / llama-tokenizer-js
  JS tokenizer for LLaMA 1 and 2
  ☆357Updated last year
• abacaj / openhermes-function-calling
  ☆135Updated last year
• mzbac / mlx-lora
  ☆38Updated last year
• tairov / QStarLearning.mojo
  ☆111Updated last year
• whitphx / transformers.js.py
  ☆87Updated 2 weeks ago
• IntrinsicLabsAI / gbnfgen
  TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces
  ☆140Updated last year
• granawkins / latent-dictionary
  A dictionary, but it shows you position in embedding space relative to some synonyms/antonyms instead of a definition.
  ☆74Updated 7 months ago
• bananaml / potassium
  An HTTP serving framework by Banana
  ☆101Updated last year
• hrishioa / wasm-ai
  Vercel and web-llm template to run wasm models directly in the browser.
  ☆160Updated last year
• argilla-io / notus
  Notus is a collection of fine-tuned LLMs using SFT, DPO, SFT+DPO, and/or any other RLHF techniques, while always keeping a data-first app…
  ☆169Updated last year
• anthropics / anthropic-tokenizer-typescript
  ☆79Updated last year
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆75Updated 2 years ago
• FanaHOVA / openai-o1-code-interpreter
  Code interpreter support for o1
  ☆32Updated 11 months ago
• do-me / SemanticFinder
  SemanticFinder - frontend-only live semantic search with transformers.js
  ☆293Updated 5 months ago
• ahoylabs / gguf.js
  A Javascript library (with Typescript types) to parse metadata of GGML based GGUF files.
  ☆50Updated last year
• nova-land / gbnf-compiler
  Plug n Play GBNF Compiler for llama.cpp
  ☆27Updated last year
• PygmalionAI / training-code
  The code we currently use to fine-tune models.
  ☆115Updated last year
• Extensible-AI / DAGent
  Build AI Agents with Your Existing Python Code!
  ☆64Updated 10 months ago
• 567-labs / fastllm
  A collection of LLM services you can self host via docker or modal labs to support your applications development
  ☆194Updated last year
• FL33TW00D / laserbeak
  Add local LLMs to your Web or Electron apps! Powered by Rust + WebGPU
  ☆103Updated 2 years ago
• keithwhor / wavtools
  Record and stream WAV audio data in the browser across all platforms
  ☆88Updated 9 months ago
• taylorai / mlx_embedding_models
  run embeddings in MLX
  ☆91Updated 11 months ago
• andreasjansson / AutoCog
  ☆40Updated 3 months ago
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆118Updated last year
• normal-computing / extended-mind-transformers
  ☆123Updated last year
• trzy / llava-cpp-server
  LLaVA server (llama.cpp).
  ☆182Updated last year
• Birch-san / mpt-play
  Command-line script for inferencing from models such as MPT-7B-Chat
  ☆100Updated 2 years ago
• SpellcraftAI / turing
  Turing machines, Rule 110, and A::B reversal using Claude 3 Opus.
  ☆58Updated last year
• teknium1 / ShareGPT-Builder
  ☆116Updated 8 months ago