joey00072 / TinyLora
Low-Rank Adaptation of Large Language Models clean implementation
☆8Updated last year
Alternatives and similar repositories for TinyLora:
Users that are interested in TinyLora are comparing it to the libraries listed below
- ☆22Updated last year
- A place to store reusable transformer components of my own creation or found on the interwebs☆48Updated last week
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆14Updated last year
- QLoRA for Masked Language Modeling☆21Updated last year
- Utilities for PyTorch distributed☆23Updated 3 weeks ago
- ☆24Updated last year
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated last year
- ☆15Updated 5 months ago
- 🚀🤗 A collection of templates for Hugging Face Spaces☆35Updated last year
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.☆17Updated last week
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- A library for squeakily cleaning and filtering language datasets.☆46Updated last year
- Describe the format of image/text datasets☆11Updated 2 years ago
- ☆31Updated 9 months ago
- A list of language models with permissive licenses such as MIT or Apache 2.0☆24Updated 2 weeks ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- Jax like function transformation engine but micro, microjax☆30Updated 4 months ago
- alternative way to calculating self attention☆18Updated 9 months ago
- ☆29Updated last year
- ☆48Updated 4 months ago
- Using multiple LLMs for ensemble Forecasting☆16Updated last year
- Code accompanying the paper "A Language Model's Guide Through Latent Space". It contains functionality for training and using concept vec…☆19Updated last year
- Tools for merging pretrained large language models.☆19Updated 9 months ago
- Example for Logging LLM Evaluator Prompt Responses☆15Updated last year
- [WIP] Transformer to embed Danbooru labelsets☆13Updated 11 months ago
- ☆12Updated 11 months ago
- Understanding the correlation between different LLM benchmarks☆29Updated last year
- Fast approximate inference on a single GPU with sparsity aware offloading☆38Updated last year
- Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…☆42Updated last year