hkproj / bert-from-scratch
BERT explained from scratch
☆12Updated last year
Related projects ⓘ
Alternatives and complementary repositories for bert-from-scratch
- A set of scripts and notebooks on LLM finetunning and dataset creation☆93Updated last month
- Mixed precision training from scratch with Tensors and CUDA☆20Updated 6 months ago
- ☆44Updated last week
- Starter pack for NeurIPS LLM Efficiency Challenge 2023.☆118Updated last year
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆47Updated 5 months ago
- Notes on quantization in neural networks☆58Updated 11 months ago
- Prune transformer layers☆64Updated 5 months ago
- Distributed training (multi-node) of a Transformer model☆43Updated 7 months ago
- This repository contains the code for dataset curation and finetuning of instruct variant of the Bilingual OpenHathi model. The resultin…☆23Updated 10 months ago
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆82Updated last year
- Notes about "Attention is all you need" video (https://www.youtube.com/watch?v=bCz4OMemCcA)☆226Updated last year
- LoRA and DoRA from Scratch Implementations☆188Updated 8 months ago
- End-to-End LLM Guide☆97Updated 4 months ago
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆86Updated last year
- LLM_library is a comprehensive repository serves as a one-stop resource hands-on code, insightful summaries.☆68Updated 10 months ago
- ☆64Updated last month
- Set of scripts to finetune LLMs☆36Updated 7 months ago
- Tutorial for how to build BERT from scratch☆83Updated 5 months ago
- RAGs: Simple implementations of Retrieval Augmented Generation (RAG) Systems☆83Updated 7 months ago
- Triton implementation of GPT/LLAMA☆16Updated 2 months ago
- This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…☆54Updated last year
- LLM Workshop by Sourab Mangrulkar☆346Updated 5 months ago
- Experimenting with small language models☆47Updated 10 months ago
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆64Updated last month
- PB-LLM: Partially Binarized Large Language Models☆148Updated last year
- Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆78Updated this week
- A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..☆166Updated 3 months ago
- Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation☆32Updated 8 months ago
- a simplified version of Meta's Llama 3 model to be used for learning☆32Updated 5 months ago
- Reference implementation of Mistral AI 7B v0.1 model.☆27Updated 10 months ago