ksm26 / Pretraining-LLMsLinks
Master the essential steps of pretraining large language models (LLMs). Learn to create high-quality datasets, configure model architectures, execute training runs, and assess model performance for efficient and effective LLM pretraining.
☆19Updated 10 months ago
Alternatives and similar repositories for Pretraining-LLMs
Users that are interested in Pretraining-LLMs are comparing it to the libraries listed below
Sorting:
- ☆83Updated last year
- Welcome to the LLMs Interview Prep Guide! This GitHub repository offers a curated set of interview questions and answers tailored for Dat…☆134Updated last year
- This repository contains a custom implementation of the BERT model, fine-tuned for specific tasks, along with an implementation of Low Ra…☆76Updated last year
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆77Updated 7 months ago
- The code for paper: Hierarchical Document Refinement for Long-context Retrieval-augmented Generation☆19Updated last week
- minimal GRPO implementation from scratch☆90Updated 2 months ago
- Fine-Tuning Llama3-8B LLM in a multi-GPU environment using DeepSpeed☆17Updated last year
- A set of scripts and notebooks on LLM finetunning and dataset creation☆111Updated 8 months ago
- Building a 2.3M-parameter LLM from scratch with LLaMA 1 architecture.☆170Updated last year
- Finetuning a codegen model with python instruction set using QLORA technique for better efficacy☆12Updated last year
- Scripts of LLM pre-training and fine-tuning (w/wo LoRA, DeepSpeed)☆80Updated last year
- Various installation guides for Large Language Models☆69Updated last month
- Starter pack for NeurIPS LLM Efficiency Challenge 2023.☆122Updated last year
- I-SHEEP: Iterative Self-enHancEmEnt Paradigm of LLMs through Self-Instruct and Self-Assessment☆16Updated 4 months ago
- This is an implementation of the paper: Searching for Best Practices in Retrieval-Augmented Generation (EMNLP2024)☆320Updated 5 months ago
- Knowledge Graph Retrieval Augmented Generation (KG-RAG) Eval Datasets☆157Updated last year
- 📚 Text Classification with LoRA (Low-Rank Adaptation) of Language Models - Efficiently fine-tune large language models for text classifi…☆48Updated last year
- Repository for the paper "MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance"☆19Updated 3 months ago
- Automation Framework using LLM-as-a-judge to Scale Eval of Gen AI solutions (RAG, Multi-turn, Query Rewrite, Text2SQL etc.); that is a go…☆27Updated 4 months ago
- Notes and commented code for RLHF (PPO)☆96Updated last year
- ☆20Updated 3 years ago
- ☆16Updated 7 months ago
- ☆9Updated 7 months ago
- All-in-One: Text Embedding, Retrieval, Reranking and RAG in Transformers☆58Updated 2 weeks ago
- Distributed training (multi-node) of a Transformer model☆68Updated last year
- [ISMB '24] Self-BioRAG: Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Models☆63Updated last year
- A notebook based tutorial series on buildling a LLM from scratch☆24Updated 8 months ago
- Collection of resources for finetuning Large Language Models (LLMs).☆83Updated 4 months ago
- Jupyter notebooks for course Building and Evaluating Advanced RAG Applications, taught by Jerry Liu (Co-founder and CEO of LlamaIndex) an…☆50Updated last year
- ☆54Updated 3 months ago