dqbd / tiktokenizer
Online playground for OpenAPI tokenizers
☆1,149Updated 3 weeks ago
Alternatives and similar repositories for tiktokenizer
Users that are interested in tiktokenizer are comparing it to the libraries listed below
Sorting:
- JS port and JS/WASM bindings for openai/tiktoken☆850Updated 3 weeks ago
- Inference engine powering open source models on OpenRouter☆916Updated 3 months ago
- Analyzing the evolution of ChatGPT's codebase through time with curated archives and scripts☆284Updated last month
- Easily share permanent links to ChatGPT conversations with your friends☆1,756Updated 7 months ago
- A framework for serving and evaluating LLM routers - save LLM costs without compromising quality☆3,935Updated 9 months ago
- Python & JS/TS SDK for running AI-generated code/code interpreting in your AI app☆1,739Updated this week
- Doing simple retrieval from LLM models at various context lengths to measure accuracy☆1,854Updated 9 months ago
- HTML to Markdown converter and crawler.☆541Updated last year
- The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4 / GPT-4o / GPT-o1. Port of OpenAI's tiktoken with…☆570Updated 2 months ago
- Fuji is an AI agent that lives in your browser's sidepanel. You can now get tasks done online with a single command!☆443Updated 5 months ago
- Template for building your own custom ChatGPT style doc search powered by Next.js, OpenAI, and Supabase.☆1,651Updated 8 months ago
- Prompt design using JSX.☆2,484Updated 3 months ago
- Data framework for your LLM applications. Focus on server side solution☆2,624Updated this week
- The Poe bot protocol☆442Updated last year
- Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)☆2,578Updated last year
- 📊 llm.report is an open-source logging and analytics platform for OpenAI: Log your ChatGPT API requests, analyze costs, and improve your…☆1,002Updated last year
- The /llms.txt file, helping language models use your website☆1,177Updated last week
- Semantic cache for LLMs. Fully integrated with LangChain and llama_index.☆7,544Updated 8 months ago
- [ICLR 2023] ReAct: Synergizing Reasoning and Acting in Language Models☆2,657Updated last year
- Common interface for interacting with AI agents. The protocol is tech stack agnostic - you can use it with any framework for building age…☆1,199Updated last month
- ☆864Updated last year
- [ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings☆1,949Updated 4 months ago
- Official Repo for ICML 2024 paper "Executable Code Actions Elicit Better LLM Agents" by Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhan…☆1,160Updated 11 months ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,700Updated this week
- Keep searching, reading webpages, reasoning until it finds the answer (or exceeding the token budget)☆4,219Updated this week
- WikiChat is an improved RAG. It stops the hallucination of large language models by retrieving data from a corpus.☆1,443Updated 2 weeks ago
- 🍰 PromptLayer - Maintain a log of your prompts and OpenAI API requests. Track, debug, and replay old completions.☆591Updated this week
- A fast tool to convert any website into LLM-ready markdown data. Built by https://supermemory.ai☆1,349Updated 9 months ago
- LiveBench: A Challenging, Contamination-Free LLM Benchmark☆732Updated this week
- OpenAI proxy on Cloudflare Worker / Vercel Edge☆494Updated last year