Alternatives and similar repositories for padding_free_llm_train:

Users that are interested in padding_free_llm_train are comparing it to the libraries listed below

• frankxwang / dpo-prefix-sharing
  DPO, but faster 🚀
  ☆34Updated 2 months ago
• BlinkDL / modded-nanogpt-rwkv
  RWKV-7: Surpassing GPT
  ☆79Updated 3 months ago
• Zyphra / Zyda_processing
  ☆31Updated 8 months ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆53Updated last week
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆57Updated 4 months ago
• Jellyfish042 / RWKV_Othello
  A specialized RWKV-7 model for Othello(a.k.a. Reversi) that predicts legal moves, evaluates positions, and performs in-context search. It…
  ☆38Updated 3 weeks ago
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆62Updated last year
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated 8 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆36Updated last year
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆40Updated 4 months ago
• lucasjinreal / ImageTokenizer
  imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…
  ☆30Updated 7 months ago
• SprocketLab / sparse_matrix_fine_tuning
  Official repository for ICML 2024 paper "MoRe Fine-Tuning with 10x Fewer Parameters"
  ☆17Updated this week
• neulab / MultiUI
  Code for Paper: Harnessing Webpage Uis For Text Rich Visual Understanding
  ☆48Updated 2 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆40Updated last year
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆24Updated 5 months ago
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆63Updated last year
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆42Updated 3 months ago
• LegallyCoder / mamba-hf
  Implementation of the Mamba SSM with hf_integration.
  ☆56Updated 5 months ago
• WENGSYX / LMTuner
  LMTuner: Make the LLM Better for Everyone
  ☆33Updated last year
• xhan77 / in-context-alignment
  In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning
  ☆33Updated last year
• jdf-prog / LLM-Engines
  ☆42Updated this week
• Infini-AI-Lab / S2FT
  ☆12Updated last month
• SparksJoe / Prism
  A Framework for Decoupling and Assessing the Capabilities of VLMs
  ☆40Updated 7 months ago
• IST-DASLab / RoSA
  Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
  ☆38Updated last year
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆96Updated 4 months ago
• PiotrNawrot / nano-sparse-attention
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆57Updated 3 weeks ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆38Updated 11 months ago
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated 8 months ago