1rgs / tokenwiz
A clone of OpenAI's Tokenizer page for HuggingFace Models
☆45Updated last year
Alternatives and similar repositories for tokenwiz:
Users that are interested in tokenwiz are comparing it to the libraries listed below
- Using modal.com to process FineWeb-edu data☆20Updated 2 weeks ago
- utilities for loading and running text embeddings with onnx☆44Updated 7 months ago
- Command-line script for inferencing from models such as LLaMA, in a chat scenario, with LoRA adaptations☆33Updated last year
- 🍳 AyaMCooking is a Voice-to-Voice Mutli-lingual RAG Agent that makes a perfect sous chef for your kitchen, in upto 10 Languages 🤌🧑🍳☆21Updated 4 months ago
- Gradio UI for a Cog API☆66Updated 11 months ago
- a version of baby agi using dspy and typed predictors☆17Updated last year
- Cog wrapper for collabora/WhisperSpeech☆25Updated last year
- LLM plugin for models hosted by Anyscale Endpoints☆33Updated 10 months ago
- ☆22Updated last year
- DSPy program/pipeline inspector widget for Jupyter/VSCode Notebooks.☆32Updated last year
- Chrome Extension for exploring Hugging Face datasets 🔎☆49Updated 6 months ago
- GPU accelerated client-side embeddings for vector search, RAG etc.☆66Updated last year
- Embedding models from Jina AI☆58Updated last year
- [WIP] AI Try-On plugin for Chrome☆27Updated last year
- Run Python functions on desktop, mobile, web, and in the cloud. https://fxn.ai/explore☆40Updated last week
- implementation of https://arxiv.org/pdf/2312.09299☆20Updated 8 months ago
- assign color hues to a collection of text fragments based on embeddings☆20Updated 9 months ago
- QLoRA for Masked Language Modeling☆21Updated last year
- Latent Large Language Models☆17Updated 6 months ago
- NLP with Rust for Python 🦀🐍☆61Updated 9 months ago
- Tools for formatting large language model prompts.☆12Updated last year
- 📝 Reference-Free automatic summarization evaluation with potential hallucination detection☆100Updated last year
- Apps that run on modal.com☆12Updated 9 months ago
- Run embedding models using ONNX☆31Updated last year
- Synthetic data derived by templating, few shot prompting, transformations on public domain corpora, and monte carlo tree search.☆31Updated 2 weeks ago
- [WIP] Transformer to embed Danbooru labelsets☆13Updated 11 months ago
- ☆31Updated last year