Alternatives and similar repositories for tokenwiz

Users that are interested in tokenwiz are comparing it to the libraries listed below

• enjalot / latent-data-modal
  Using modal.com to process FineWeb-edu data
  ☆20Updated 7 months ago
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated 2 months ago
• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆65Updated last year
• deployradiant / pychatml
  Chat Markup Language conversation library
  ☆55Updated last year
• simonw / llm-embed-jina
  Embedding models from Jina AI
  ☆65Updated last year
• cfahlgren1 / hf-data-explorer
  Chrome Extension for exploring Hugging Face datasets 🔎
  ☆49Updated last year
• nateraw / modal-examples
  Apps that run on modal.com
  ☆12Updated 2 months ago
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆93Updated last month
• multimodalart / grog
  Gradio UI for a Cog API
  ☆69Updated last year
• simonw / llm-anyscale-endpoints
  LLM plugin for models hosted by Anyscale Endpoints
  ☆35Updated last year
• smol-ai / llama-fine-tuning-hackameetup
  The official repository for the Anything But Wrappers: Llama Edition Hackameetup
  ☆22Updated 2 years ago
• lucataco / cog-whisperspeech
  Cog wrapper for collabora/WhisperSpeech
  ☆24Updated last year
• replicate / hype
  A feed of trending repos/models from GitHub, Replicate, HuggingFace, and Reddit.
  ☆135Updated 5 months ago
• official-elinas / zeus-llm-trainer
  Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models
  ☆69Updated 2 years ago
• Birch-san / llama-play
  Command-line script for inferencing from models such as LLaMA, in a chat scenario, with LoRA adaptations
  ☆33Updated 2 years ago
• mrcolo / longboii
  ☆19Updated 2 years ago
• Muhtasham / summarization-eval
  📝 Reference-Free automatic summarization evaluation with potential hallucination detection
  ☆102Updated last year
• FanaHOVA / podcast-summarizer
  OpenAI Whisper + davinci for podcast summarization
  ☆71Updated 2 years ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆58Updated 3 weeks ago
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆118Updated last year
• Narsil / hf-chat
  ☆25Updated 11 months ago
• BBischof / yapping
  Verbosity control for AI agents
  ☆64Updated last year
• teknium1 / transformers-gptq-quant
  ☆45Updated 2 years ago
• kookaburracodes / investor-education-chatchain
  Not financial advice.
  ☆27Updated 2 years ago
• jacoblee93 / oss-model-extraction-evals
  ☆30Updated last year
• louisbrulenaudet / ragoon
  High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡
  ☆68Updated last year
• raga-ai-hub / ragaai-catalyst-v1
  ☆31Updated 9 months ago
• iulia-b10 / multilingual-embedding-models
  ☆20Updated last year
• virevolai / logos-shift-client
  Replace expensive LLM calls with finetunes automatically
  ☆64Updated last year
• Birch-san / falcon-play
  Command-line script for inferencing from models such as falcon-7b-instruct
  ☆74Updated 2 years ago