cindysridykhan / instruct_storyteller_tinyllama2
Training and Fine-tuning an llm in Python and PyTorch.
☆38Updated last year
Related projects: ⓘ
- experiments with inference on llama☆106Updated 3 months ago
- ☆85Updated 7 months ago
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆115Updated 5 months ago
- This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…☆47Updated 11 months ago
- ☆109Updated last month
- ☆92Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆158Updated 2 months ago
- Small and Efficient Mathematical Reasoning LLMs☆69Updated 7 months ago
- Fully fine-tune large models like Mistral, Llama-2-13B, or Qwen-14B completely for free☆217Updated 6 months ago
- Tune MPTs☆84Updated last year
- ☆73Updated 8 months ago
- A pipeline for LLM knowledge distillation☆68Updated last month
- Low-Rank adapter extraction for fine-tuned transformers model☆154Updated 4 months ago
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆107Updated last year
- Set of scripts to finetune LLMs☆36Updated 5 months ago
- LLaMA 2 implemented from scratch in PyTorch☆216Updated 11 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆258Updated 10 months ago
- ☆75Updated 3 weeks ago
- From scratch implementation of a vision language model in pure PyTorch☆149Updated 4 months ago
- RWKV in nanoGPT style☆170Updated 3 months ago
- Merge Transformers language models by use of gradient parameters.☆193Updated last month
- Model, Code & Data for the EMNLP'23 paper "Making Large Language Models Better Data Creators"☆107Updated 11 months ago
- An Open Source Toolkit For LLM Distillation☆284Updated last month
- RAFT, or Retrieval-Augmented Fine-Tuning, is a method comprising of a fine-tuning and a RAG-based retrieval phase. It is particularly sui…☆60Updated 3 weeks ago
- GPTQLoRA: Efficient Finetuning of Quantized LLMs with GPTQ☆96Updated last year
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆87Updated this week
- Multipack distributed sampler for fast padding-free training of LLMs☆170Updated last month
- Data preparation code for Amber 7B LLM☆76Updated 4 months ago
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆195Updated 4 months ago
- QLoRA: Efficient Finetuning of Quantized LLMs☆74Updated 5 months ago