IBM / dolomite-engine
Dolomite Engine is a library for pretraining/finetuning LLMs
☆36Updated this week
Alternatives and similar repositories for dolomite-engine:
Users that are interested in dolomite-engine are comparing it to the libraries listed below
- ☆48Updated last year
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆65Updated 6 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆36Updated last year
- ☆25Updated last year
- ☆78Updated 10 months ago
- some common Huggingface transformers in maximal update parametrization (µP)☆78Updated 2 years ago
- Triton Implementation of HyperAttention Algorithm☆46Updated last year
- ☆38Updated 10 months ago
- ☆47Updated last year
- Experiments for efforts to train a new and improved t5☆77Updated 10 months ago
- ☆42Updated last year
- ☆51Updated 9 months ago
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆56Updated 8 months ago
- A toolkit for scaling law research ⚖☆47Updated 3 weeks ago
- ☆20Updated last year
- Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"☆21Updated 5 months ago
- codebase release for EMNLP2023 paper publication☆19Updated 11 months ago
- 🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.☆34Updated this week
- ☆51Updated 5 months ago
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆57Updated 3 weeks ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆40Updated last year
- ☆49Updated 7 months ago
- Repository for "I am a Strange Dataset: Metalinguistic Tests for Language Models"☆41Updated last year
- BPE modification that implements removing of the intermediate tokens during tokenizer training.☆25Updated 2 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆54Updated 5 months ago
- Train, tune, and infer Bamba model☆84Updated last month
- A repository for research on medium sized language models.☆76Updated 8 months ago
- a pipeline for using api calls to agnostically convert unstructured data into structured training data☆29Updated 4 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆59Updated 2 months ago