hkproj / bert-from-scratchLinks
BERT explained from scratch
☆16Updated 2 years ago
Alternatives and similar repositories for bert-from-scratch
Users that are interested in bert-from-scratch are comparing it to the libraries listed below
Sorting:
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆48Updated last year
- ☆99Updated last year
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆117Updated 2 years ago
- ☆45Updated 6 months ago
- Distributed training (multi-node) of a Transformer model☆87Updated last year
- A set of scripts and notebooks on LLM finetunning and dataset creation☆111Updated last year
- Unofficial implementation of https://arxiv.org/pdf/2407.14679☆51Updated last year
- Prune transformer layers☆74Updated last year
- Starter pack for NeurIPS LLM Efficiency Challenge 2023.☆128Updated 2 years ago
- This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…☆74Updated 2 years ago
- Notes on quantization in neural networks☆109Updated last year
- Notes about LLaMA 2 model☆70Updated 2 years ago
- Notes on Direct Preference Optimization☆23Updated last year
- LLM Workshop by Sourab Mangrulkar☆397Updated last year
- LLaMA 2 implemented from scratch in PyTorch☆361Updated 2 years ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆257Updated 2 years ago
- An extension of the nanoGPT repository for training small MOE models.☆215Updated 8 months ago
- ☆86Updated last year
- Code for studying the super weight in LLM☆121Updated 11 months ago
- Building GPT ...☆18Updated 11 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆347Updated 6 months ago
- From scratch implementation of a vision language model in pure PyTorch☆251Updated last year
- LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's architecture in a simpler manner.☆190Updated last year
- Notes about "Attention is all you need" video (https://www.youtube.com/watch?v=bCz4OMemCcA)☆327Updated 2 years ago
- Mixed precision training from scratch with Tensors and CUDA☆28Updated last year
- Project 2 (Building Large Language Models) for Stanford CS324: Understanding and Developing Large Language Models (Winter 2022)☆105Updated 2 years ago
- ☆189Updated last year
- LoRA and DoRA from Scratch Implementations☆215Updated last year
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆181Updated 2 weeks ago
- A curated reading list of research in Adaptive Computation, Inference-Time Computation & Mixture of Experts (MoE).☆160Updated 10 months ago