joey00072 / TinyLora
Low-Rank Adaptation of Large Language Models clean implementation
☆8Updated last year
Alternatives and similar repositories for TinyLora:
Users that are interested in TinyLora are comparing it to the libraries listed below
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆14Updated last year
- Jax like function transformation engine but micro, microjax☆30Updated 4 months ago
- ☆22Updated last year
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated last year
- Implementation of SelfExtend from the paper "LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning" from Pytorch and Zeta☆13Updated 4 months ago
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.☆17Updated this week
- ☆54Updated last year
- ☆15Updated 5 months ago
- A list of language models with permissive licenses such as MIT or Apache 2.0☆24Updated last week
- alternative way to calculating self attention☆18Updated 9 months ago
- A sample pattern for running CI tests on Modal☆15Updated 5 months ago
- QLoRA for Masked Language Modeling☆21Updated last year
- Tools for merging pretrained large language models.☆19Updated 9 months ago
- Exploration using DSPy to optimize modules to maximize performance on the OpenToM dataset☆14Updated last year
- Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response format☆27Updated last year
- LLM training in simple, raw C/CUDA☆14Updated 3 months ago
- A place to store reusable transformer components of my own creation or found on the interwebs☆47Updated 2 weeks ago
- ☆48Updated 4 months ago
- Using multiple LLMs for ensemble Forecasting☆16Updated last year
- ☆29Updated last year
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆25Updated 4 months ago
- A clone of OpenAI's Tokenizer page for HuggingFace Models☆45Updated last year
- BH hackathon☆14Updated 11 months ago
- ☆12Updated 11 months ago
- implementation of https://arxiv.org/pdf/2312.09299☆20Updated 8 months ago
- ☆20Updated last year
- ☆24Updated last year
- ☆14Updated last year