sekstini / basedxl
☆15Updated 6 months ago
Related projects: ⓘ
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆62Updated last year
- QuIP quantization☆41Updated 6 months ago
- ☆18Updated last month
- ☆40Updated 2 months ago
- SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia☆40Updated last year
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆27Updated last month
- ☆22Updated 3 months ago
- Here we will test various linear attention designs.☆55Updated 4 months ago
- [WIP] Transformer to embed Danbooru labelsets☆13Updated 5 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆34Updated 10 months ago
- GoldFinch and other hybrid transformer components☆38Updated 2 months ago
- Simple and fast low-bit matmul kernels in CUDA☆48Updated this week
- This repository contains code for the MicroAdam paper.☆9Updated 2 months ago
- PyTorch half precision gemm lib w/ fused optional bias + optional relu/gelu☆25Updated 3 weeks ago
- ☆50Updated last month
- implementation of https://arxiv.org/pdf/2312.09299☆19Updated 2 months ago
- ☆48Updated 6 months ago
- RWKV model implementation☆38Updated last year
- Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)☆22Updated 3 months ago
- ☆68Updated 2 months ago
- A repository for research on medium sized language models.☆71Updated 3 months ago
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆14Updated 11 months ago
- Triton Implementation of HyperAttention Algorithm☆46Updated 9 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆36Updated 8 months ago
- Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business /…☆40Updated last year
- Make triton easier☆39Updated 3 months ago
- Repository for CPU Kernel Generation for LLM Inference☆25Updated last year
- ☆66Updated 3 months ago
- Collection of autoregressive model implementation☆62Updated 2 weeks ago
- BigKnow2022: Bringing Language Models Up to Speed☆13Updated last year