C-J-Cundy / gpt4-tokenizer
Hosting the JSON for the GPT4 Tokenizer
☆65Updated last year
Related projects ⓘ
Alternatives and complementary repositories for gpt4-tokenizer
- The GitHub repo for Goal Driven Discovery of Distributional Differences via Language Descriptions☆68Updated last year
- Code repository for the c-BTM paper☆105Updated last year
- Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response format☆27Updated last year
- Reimplementation of the task generation part from the Alpaca paper☆119Updated last year
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…☆34Updated last year
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆25Updated last year
- Public Inflection Benchmarks☆69Updated 8 months ago
- ☆46Updated this week
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- Advanced Reasoning Benchmark Dataset for LLMs☆45Updated last year
- ☆86Updated 5 months ago
- ☆48Updated last year
- Track the progress of LLM context utilisation☆53Updated 4 months ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆32Updated last year
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆23Updated 8 months ago
- ☆22Updated last year
- Lightweight tools for quick and easy LLM demo's☆26Updated last month
- Apps built using Inspired Cognition's Critique.☆58Updated last year
- ☆31Updated last year
- Repository for "I am a Strange Dataset: Metalinguistic Tests for Language Models"☆39Updated 10 months ago
- ☆24Updated last year
- Code of ICLR paper: https://openreview.net/forum?id=-cqvvvb-NkI☆91Updated last year
- Do Multilingual Language Models Think Better in English?☆41Updated last year
- Probabilistic LLM evaluations. [CogSci2023; ACL2023]☆72Updated 3 months ago
- Experiments with generating opensource language model assistants☆97Updated last year
- Evaluating LLMs with CommonGen-Lite☆85Updated 8 months ago
- Using open source LLMs to build synthetic datasets for direct preference optimization☆40Updated 8 months ago
- Functional Benchmarks and the Reasoning Gap☆78Updated last month