C-J-Cundy / gpt4-tokenizer
Hosting the JSON for the GPT4 Tokenizer
☆65Updated last year
Alternatives and similar repositories for gpt4-tokenizer:
Users that are interested in gpt4-tokenizer are comparing it to the libraries listed below
- ☆48Updated last year
- ☆22Updated last year
- ☆92Updated 3 weeks ago
- Reimplementation of the task generation part from the Alpaca paper☆119Updated last year
- The GitHub repo for Goal Driven Discovery of Distributional Differences via Language Descriptions☆68Updated last year
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- ☆34Updated last year
- ☆79Updated last week
- A library for squeakily cleaning and filtering language datasets.☆45Updated last year
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated 10 months ago
- Lightweight tools for quick and easy LLM demo's☆26Updated 3 months ago
- Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA☆101Updated 5 months ago
- ☆24Updated last year
- Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response format☆27Updated last year
- Experiments for efforts to train a new and improved t5☆77Updated 9 months ago
- ☆46Updated 2 months ago
- Advanced Reasoning Benchmark Dataset for LLMs☆45Updated last year
- Public Inflection Benchmarks☆69Updated 10 months ago
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆25Updated last year
- Code of ICLR paper: https://openreview.net/forum?id=-cqvvvb-NkI☆91Updated last year
- Probabilistic LLM evaluations. [CogSci2023; ACL2023]☆72Updated 5 months ago
- Official code for ACL 2023 (short, findings) paper "Recursion of Thought: A Divide and Conquer Approach to Multi-Context Reasoning with L…☆42Updated last year
- Command-line script for inferencing from models such as MPT-7B-Chat☆101Updated last year
- ☆67Updated 5 months ago
- QLoRA with Enhanced Multi GPU Support☆36Updated last year
- Based on the tree of thoughts paper☆46Updated last year
- [WIP] Transformer to embed Danbooru labelsets☆13Updated 9 months ago
- Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models☆70Updated last year