Alternatives and similar repositories for QLoRA-for-MLM:

Users that are interested in QLoRA-for-MLM are comparing it to the libraries listed below

• Birch-san / booru-embed
  [WIP] Transformer to embed Danbooru labelsets
  ☆13Updated 11 months ago
• CarperAI / treasure_trove
  ☆22Updated last year
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• argilla-io / distilabel-spin-dibt
  Repository containing the SPIN experiments on the DIBT 10k ranked prompts
  ☆24Updated 11 months ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆46Updated last year
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆63Updated last year
• SebastianBodza / EnsembleForecasting
  Using multiple LLMs for ensemble Forecasting
  ☆16Updated last year
• Zyphra / Zyda_processing
  ☆31Updated 8 months ago
• somewheresystems / llama2mlx
  Karpathy's llama2.c transpiled to MLX for Apple Silicon
  ☆15Updated last year
• CarperAI / decontamination
  This repository contains code for cleaning your training data of benchmark data to help combat data snooping.
  ☆25Updated last year
• arcee-ai / DAM
  ☆48Updated 4 months ago
• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆41Updated 11 months ago
• pacman100 / peft-codegen-25
  ☆24Updated last year
• ElleLeonne / Lightning-ReLoRA
  A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.
  ☆33Updated last year
• huggingface / peft-pytorch-conference
  Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…
  ☆14Updated last year
• nbroad1881 / strideformer
  Using short models to classify long texts
  ☆21Updated 2 years ago
• emrgnt-cmplxty / SmolTrainer
  ☆20Updated last year
• pchizhov / picky_bpe
  BPE modification that implements removing of the intermediate tokens during tokenizer training.
  ☆25Updated 3 months ago
• ChrisHayduk / qlora-multi-gpu
  QLoRA with Enhanced Multi GPU Support
  ☆36Updated last year
• KaiNylund / lm-weights-encode-time
  ☆67Updated 6 months ago
• krypticmouse / matryoshka-representation-learning
  PyTorch implementation for MRL
  ☆18Updated last year
• YuchenJin / llm.c
  LLM training in simple, raw C/CUDA
  ☆14Updated 3 months ago
• kevinwu23 / StanfordFineTuneBench
  ☆27Updated 3 months ago
• teknium1 / transformers-gptq-quant
  ☆48Updated last year
• aymeric-roucher / LongContext_vs_RAG_NeedleInAHaystack
  Comparing retrieval abilities from GPT4-Turbo and a RAG system on a toy example for various context lengths
  ☆35Updated last year
• huggingface / huggingface-inference-toolkit
  Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.
  ☆63Updated this week