Alternatives and similar repositories for llama-tokenizer-js

Users that are interested in llama-tokenizer-js are comparing it to the libraries listed below

• okuvshynov / slowllama
  Finetune llama2-70b and codellama on MacBook Air without quantization
  ☆447Updated last year
• lastmile-ai / llama-retrieval-plugin
  LLaMa retrieval plugin script using OpenAI's retrieval plugin
  ☆325Updated 2 years ago
• r2d4 / openlm
  OpenAI-compatible Python client that can call any LLM
  ☆371Updated 2 years ago
• IntrinsicLabsAI / gbnfgen
  TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces
  ☆137Updated 11 months ago
• togethercomputer / redpajama.cpp
  Extend the original llama.cpp repo to support redpajama model.
  ☆118Updated 9 months ago
• dmarcos / llama2.c-web
  Simple repo that compiles and runs llama2.c on the Web
  ☆57Updated last year
• lxe / wasm-gpt
  Tensor library for machine learning
  ☆276Updated 2 years ago
• belladoreai / llama3-tokenizer-js
  JS tokenizer for LLaMA 3 and LLaMA 3.1
  ☆110Updated 3 months ago
• valine / NeuralFlow
  Visualize the intermediate output of Mistral 7B
  ☆367Updated 5 months ago
• automorphic-ai / trex
  Enforce structured output from LLMs 100% of the time
  ☆249Updated 11 months ago
• ColinEberhardt / langchain-mini
  🦜️🔗 This is a very simple re-implementation of LangChain, in ~100 lines of code
  ☆253Updated last year
• cognitivecomputations / OpenChatML
  ☆157Updated 11 months ago
• bigcode-project / starcoder.cpp
  C++ implementation for 💫StarCoder
  ☆453Updated last year
• simonmesmith / agentflow
  Complex LLM Workflows from Simple JSON.
  ☆304Updated last year
• FL33TW00D / laserbeak
  Add local LLMs to your Web or Electron apps! Powered by Rust + WebGPU
  ☆102Updated 2 years ago
• newhouseb / clownfish
  Constrained Decoding for LLMs against JSON Schema
  ☆327Updated 2 years ago
• eugenepentland / landmark-attention-qlora
  Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA
  ☆123Updated 2 years ago
• replicate / cog-llama-template
  LLaMA Cog template
  ☆307Updated last year
• declare-lab / flan-alpaca
  This repository contains code for extending the Stanford Alpaca synthetic instruction tuning to existing instruction-tuned models such as…
  ☆352Updated last year
• Birch-san / falcon-play
  Command-line script for inferencing from models such as falcon-7b-instruct
  ☆75Updated 2 years ago
• do-me / SemanticFinder
  SemanticFinder - frontend-only live semantic search with transformers.js
  ☆278Updated 2 months ago
• redotvideo / haven
  LLM fine-tuning and eval
  ☆346Updated last year
• SkunkworksAI / hydra-moe
  ☆415Updated last year
• abacaj / fine-tune-mistral
  Fine-tune mistral-7B on 3090s, a100s, h100s
  ☆714Updated last year
• jondurbin / airoboros
  Customizable implementation of the self-instruct paper.
  ☆1,045Updated last year
• chris-alexiuk / alpaca-lora
  Instruct-tune LLaMA on consumer hardware
  ☆362Updated 2 years ago
• zphang / minimal-llama
  ☆458Updated last year
• migtissera / Sensei
  Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI
  ☆222Updated last year
• aigoopy / llm-jeopardy
  Automated prompting and scoring framework to evaluate LLMs using updated human knowledge prompts
  ☆110Updated last year
• linonetwo / langchain-alpaca
  Run Alpaca LLM in LangChain
  ☆218Updated last year