iantbutler01 / ditty
A library for simplifying fine tuning with multi gpu setups in the Huggingface ecosystem.
☆15Updated 3 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for ditty
- Training hybrid models for dummies.☆15Updated 3 weeks ago
- Implementation of SelfExtend from the paper "LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning" from Pytorch and Zeta☆13Updated last week
- Minimum Description Length probing for neural network representations☆16Updated last week
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated 8 months ago
- implementation of https://arxiv.org/pdf/2312.09299☆19Updated 4 months ago
- PyTorch Implementation of the paper "MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training"☆23Updated last week
- Efficient Dictionary Learning with Switch Sparse Autoencoders (SAEs)☆12Updated last month
- Implementation of Spectral State Space Models☆17Updated 8 months ago
- Github repo for Peifeng's internship project☆12Updated last year
- Latent Large Language Models☆16Updated 2 months ago
- GoldFinch and other hybrid transformer components☆39Updated 4 months ago
- [ICML 2023] "Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation", Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal, …☆37Updated last year
- NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer☆37Updated 7 months ago
- Code for "Accelerating Training with Neuron Interaction and Nowcasting Networks"☆14Updated 2 weeks ago
- High-performance tokenized language data-loader for Python C++ extension☆12Updated 3 months ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- Trying to deconstruct RWKV in understandable terms☆14Updated last year
- Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"☆15Updated last week
- Submission to the inverse scaling prize☆23Updated last year
- Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification☆11Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆18Updated last year
- ☆32Updated last year
- Q-Probe: A Lightweight Approach to Reward Maximization for Language Models☆37Updated 5 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆23Updated 8 months ago
- LLMs as Collaboratively Edited Knowledge Bases☆43Updated 9 months ago
- RWKV model implementation☆38Updated last year
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- ☆43Updated 4 months ago
- this is for fun, ain't it grand!☆12Updated 7 months ago