Alternatives and similar repositories for llama3-tokenizer-js:

Users that are interested in llama3-tokenizer-js are comparing it to the libraries listed below

• IntrinsicLabsAI / gbnfgen
  TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces
  ☆131Updated 6 months ago
• belladoreai / llama-tokenizer-js
  JS tokenizer for LLaMA 1 and 2
  ☆348Updated 7 months ago
• do-me / SemanticFinder
  SemanticFinder - frontend-only live semantic search with transformers.js
  ☆250Updated last month
• migtissera / Sensei
  Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI
  ☆222Updated 9 months ago
• tangledgroup / llama-cpp-wasm
  WebAssembly (Wasm) Build and Bindings for llama.cpp
  ☆231Updated 6 months ago
• sshh12 / multi_token
  Embed arbitrary modalities (images, audio, documents, etc) into large language models.
  ☆177Updated 10 months ago
• Shubin123 / kokorojs
  kokoro text to speech using javascript
  ☆50Updated this week
• Birch-san / falcon-play
  Command-line script for inferencing from models such as falcon-7b-instruct
  ☆76Updated last year
• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆66Updated last year
• epolewski / EricLLM
  A fast batching API to serve LLM models
  ☆180Updated 9 months ago
• thooton / muse
  Let's create synthetic textbooks together :)
  ☆73Updated last year
• zya / litellmjs
  JavaScript implementation of LiteLLM.
  ☆108Updated last week
• pdufour / llm-coder
  In-browser LLM website generator
  ☆33Updated this week
• rahuldshetty / llm.js
  Run Large-Language Models (LLMs) 🚀 directly in your browser!
  ☆175Updated 4 months ago
• cognitivecomputations / OpenChatML
  ☆151Updated 6 months ago
• thomasgauthier / LoRD
  Low-Rank adapter extraction for fine-tuned transformers models
  ☆167Updated 8 months ago
• ahoylabs / gguf.js
  A Javascript library (with Typescript types) to parse metadata of GGML based GGUF files.
  ☆45Updated 6 months ago
• itsme2417 / PolyMind
  A multimodal, function calling powered LLM webui.
  ☆213Updated 4 months ago
• yusufhilmi / client-vector-search
  A client side vector search library that can embed, store, search, and cache vectors. Works on the browser and node. It outperforms OpenA…
  ☆179Updated 8 months ago
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆118Updated last year
• pranavjad / tinyllama-bitnet
  Train your own small bitnet model
  ☆64Updated 3 months ago
• Maximilian-Winter / llama_cpp_function_calling
  ☆31Updated last year
• bdambrosio / AllTheWorldAPlay
  All the world is a play, we are but actors in it.
  ☆47Updated this week
• bwasti / btb
  Blue-text Bot AI. Uses Ollama + AppleScript
  ☆50Updated 8 months ago
• av / klmbr
  klmbr - a prompt pre-processing technique to break through the barrier of entropy while generating text with LLMs
  ☆67Updated 4 months ago
• p-e-w / arrows
  A frontend for creative writing with LLMs
  ☆115Updated 6 months ago
• Mihaiii / backtrack_sampler
  An easy-to-understand framework for LLM samplers that rewind and revise generated tokens
  ☆120Updated this week
• runpod-workers / worker-vllm
  The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
  ☆273Updated this week
• cognitivecomputations / grokadamw
  ☆122Updated 5 months ago
• dmarcos / llama2.c-web
  Simple repo that compiles and runs llama2.c on the Web
  ☆54Updated last year