Alternatives and similar repositories for llama-tokenizer-js:

Users that are interested in llama-tokenizer-js are comparing it to the libraries listed below

• IntrinsicLabsAI / gbnfgen
  TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces
  ☆135Updated 8 months ago
• lastmile-ai / llama-retrieval-plugin
  LLaMa retrieval plugin script using OpenAI's retrieval plugin
  ☆324Updated last year
• migtissera / Sensei
  Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI
  ☆224Updated 10 months ago
• ColinEberhardt / langchain-mini
  🦜️🔗 This is a very simple re-implementation of LangChain, in ~100 lines of code
  ☆253Updated last year
• r2d4 / openlm
  OpenAI-compatible Python client that can call any LLM
  ☆371Updated last year
• newhouseb / clownfish
  Constrained Decoding for LLMs against JSON Schema
  ☆326Updated last year
• okuvshynov / slowllama
  Finetune llama2-70b and codellama on MacBook Air without quantization
  ☆448Updated 11 months ago
• kayvr / token-hawk
  WebGPU LLM inference tuned by hand
  ☆149Updated last year
• cognitivecomputations / OpenChatML
  ☆152Updated 8 months ago
• togethercomputer / redpajama.cpp
  Extend the original llama.cpp repo to support redpajama model.
  ☆117Updated 6 months ago
• valine / NeuralFlow
  Visualize the intermediate output of Mistral 7B
  ☆344Updated 2 months ago
• taprosoft / llm_finetuning
  Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…
  ☆147Updated last year
• FL33TW00D / laserbeak
  Add local LLMs to your Web or Electron apps! Powered by Rust + WebGPU
  ☆102Updated last year
• persimmon-ai-labs / adept-inference
  Inference code for Persimmon-8B
  ☆415Updated last year
• automorphic-ai / trex
  Enforce structured output from LLMs 100% of the time
  ☆248Updated 8 months ago
• aigoopy / llm-jeopardy
  Automated prompting and scoring framework to evaluate LLMs using updated human knowledge prompts
  ☆111Updated last year
• lxe / wasm-gpt
  Tensor library for machine learning
  ☆278Updated last year
• Birch-san / falcon-play
  Command-line script for inferencing from models such as falcon-7b-instruct
  ☆76Updated last year
• umuthopeyildirim / DOOM-Mistral
  Mistral7B playing DOOM
  ☆130Updated 8 months ago
• eugenepentland / landmark-attention-qlora
  Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA
  ☆123Updated last year
• SkunkworksAI / hydra-moe
  ☆412Updated last year
• do-me / SemanticFinder
  SemanticFinder - frontend-only live semantic search with transformers.js
  ☆263Updated 3 months ago
• llmonpy / needle-in-a-needlestack
  ☆111Updated last month
• declare-lab / flan-alpaca
  This repository contains code for extending the Stanford Alpaca synthetic instruction tuning to existing instruction-tuned models such as…
  ☆351Updated last year
• rogeriochaves / driver
  ☆125Updated last year
• token-js / token.js
  Integrate 200+ LLMs with one TypeScript SDK using OpenAI's format.
  ☆192Updated 3 weeks ago
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆93Updated last year
• mithril-security / blind_chat
  A fully in-browser privacy solution to make Conversational AI privacy-friendly
  ☆227Updated 5 months ago
• team-openpm / openpm
  ☆142Updated last year
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated 7 months ago