Alternatives and similar repositories for Position-Prediction-Pretraining:

Users that are interested in Position-Prediction-Pretraining are comparing it to the libraries listed below

• jason9693 / FROZEN
  ☆13Updated 2 years ago
• formll / resolving-scaling-law-discrepancies
  ☆16Updated 6 months ago
• wzq016 / PINE
  Offcial Repo of Paper "Eliminating Position Bias of Language Models: A Mechanistic Approach""
  ☆11Updated 4 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆23Updated 6 months ago
• IdoAmos / not-from-scratch
  ☆26Updated 2 months ago
• alextamkin / active-learning-pretrained-models
  Active Learning Helps Pretrained Models Learn the Intended Task (https://arxiv.org/abs/2204.08491) by Alex Tamkin, Dat Nguyen, Salil Desh…
  ☆11Updated 2 years ago
• sc782 / SBM-Transformer
  ☆13Updated last year
• abhishekpanigrahi1996 / transformer_in_transformer
  ☆44Updated last year
• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆35Updated 7 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆52Updated 5 months ago
• jadehaus / preference-flow-matching
  Official code implementation for the work Preference Alignment with Flow Matching (NeurIPS 2024)
  ☆20Updated 2 months ago
• ElisaNguyen / bayesian-tda
  Code for NeurIPS'23 paper "A Bayesian Approach To Analysing Training Data Attribution In Deep Learning"
  ☆15Updated last year
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆33Updated 3 months ago
• dmis-lab / Monet
  Monet: Mixture of Monosemantic Experts for Transformers
  ☆44Updated this week
• jiyounglee-0523 / FourierDecoder
  Official repository for Fourier model that can generate periodic signals
  ☆9Updated 2 years ago
• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆23Updated 2 months ago
• LooperXX / ManagerTower
  Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning
  ☆11Updated 3 weeks ago
• lsj2408 / URPE
  [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)
  ☆34Updated last year
• PKU-ML / non_neg
  Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning
  ☆38Updated 8 months ago
• fangyuan-ksgk / selective-attention-transformer
  Unofficial Implementation of Selective Attention Transformer
  ☆14Updated 2 months ago
• duykhuongnguyen / LASeR-MAB
  Code for paper: "LASeR: Learning to Adaptively Select Reward Models with Multi-Arm Bandits"
  ☆13Updated 3 months ago
• zhangir-azerbayev / MetaMath
  ☆11Updated last year
• janphilippfranken / sami
  Self-Supervised Alignment with Mutual Information
  ☆16Updated 7 months ago
• LukasHedegaard / structured-pruning-adapters
  Structured Pruning Adapters in PyTorch
  ☆16Updated last year
• kaistAI / GAP
  [ACL 2023] Gradient Ascent Post-training Enhances Language Model Generalization
  ☆29Updated 4 months ago
• uclaml / MoE
  Towards Understanding the Mixture-of-Experts Layer in Deep Learning
  ☆22Updated last year
• Cranial-XIX / longhorn
  Official PyTorch Implementation of the Longhorn Deep State Space Model
  ☆45Updated last month
• locuslab / T-MARS
  Code for T-MARS data filtering
  ☆35Updated last year
• JeanKaddour / LAWA
  Latest Weight Averaging (NeurIPS HITY 2022)
  ☆28Updated last year