fattorib / Little-GPT
GPT* - Training faster small transformers using ALiBi, Parallel Residual Connections and more!
☆23Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for Little-GPT
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆25Updated last year
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- Efficiently computing & storing token n-grams from large corpora☆15Updated last month
- Experimental sampler to make LLMs more creative☆30Updated last year
- A library for squeakily cleaning and filtering language datasets.☆45Updated last year
- ☆49Updated 8 months ago
- Simple Model Similarities Analysis☆20Updated 9 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆23Updated 8 months ago
- An experiment to see if chatgpt can improve the output of the stanford alpaca dataset☆12Updated last year
- Merge LLM that are split in to parts☆25Updated last year
- Experiments with generating opensource language model assistants☆97Updated last year
- ☆41Updated 3 weeks ago
- ☆24Updated last year
- One stop shop for all things carp☆59Updated 2 years ago
- [WIP] Transformer to embed Danbooru labelsets☆13Updated 7 months ago
- Official implementation of LoT paper: "Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic"☆18Updated 8 months ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- ☆57Updated 11 months ago
- Fast inference of Instruct tuned LLaMa on your personal devices.☆22Updated last year
- RWKV-7: Surpassing GPT☆45Updated this week
- ☆32Updated last year
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆23Updated last week
- Code for the paper-"Mirostat: A Perplexity-Controlled Neural Text Decoding Algorithm" (https://arxiv.org/abs/2007.14966).☆57Updated 2 years ago
- RWKV model implementation☆38Updated last year
- Modified Beam Search with periodical restart☆12Updated 2 months ago
- an implementation of Self-Extend, to expand the context window via grouped attention☆118Updated 10 months ago
- Latent Large Language Models☆16Updated 2 months ago
- GoldFinch and other hybrid transformer components☆39Updated 4 months ago
- QLoRA with Enhanced Multi GPU Support☆36Updated last year