arshadshk / Position-Prediction-Pretraining

Related projects ⓘ

Alternatives and complementary repositories for Position-Prediction-Pretraining

• jason9693 / FROZEN
  ☆13Updated 2 years ago
• kaistAI / GAP
  [ACL 2023] Gradient Ascent Post-training Enhances Language Model Generalization
  ☆27Updated 2 months ago
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• sc782 / SBM-Transformer
  ☆13Updated last year
• LukasHedegaard / structured-pruning-adapters
  Structured Pruning Adapters in PyTorch
  ☆15Updated last year
• wzq016 / PINE
  Offcial Repo of Paper "Eliminating Position Bias of Language Models: A Mechanistic Approach""
  ☆11Updated 2 months ago
• lsj2408 / URPE
  [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)
  ☆33Updated last year
• SamsungSAILMontreal / ghn3
  Code for "Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?" [ICML 2023]
  ☆31Updated 2 months ago
• IdoAmos / not-from-scratch
  ☆25Updated last month
• alextamkin / active-learning-pretrained-models
  Active Learning Helps Pretrained Models Learn the Intended Task (https://arxiv.org/abs/2204.08491) by Alex Tamkin, Dat Nguyen, Salil Desh…
  ☆11Updated 2 years ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆44Updated last year
• r-three / RAD
  Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model
  ☆41Updated 10 months ago
• lapisrocks / DiscreteAdversarialDistillation
  [NeurIPS 2023] Official repository for "Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models"
  ☆11Updated 5 months ago
• abhishekpanigrahi1996 / transformer_in_transformer
  ☆44Updated last year
• locuslab / T-MARS
  Code for T-MARS data filtering
  ☆35Updated last year
• gregorbachmann / scaling_mlps
  ☆51Updated 5 months ago
• fredzzhang / atlas
  Official PyTorch implementation for NeurIPS'24 paper "Knowledge Composition using Task Vectors with Learned Anisotropic Scaling"
  ☆11Updated 2 weeks ago
• formll / resolving-scaling-law-discrepancies
  ☆15Updated 4 months ago
• ExplainableML / fomo_in_flux
  Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]
  ☆35Updated 2 months ago
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆36Updated 2 years ago
• alinlab / HOMER
  Official implementation of Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs (ICLR 2024).
  ☆32Updated 3 months ago
• joeljang / FLM
  All-in-one repository for Fine-tuning & Pretraining (Large) Language Models
  ☆15Updated last year
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆23Updated 5 months ago
• 1202kbs / EBCLR
  Official PyTorch implementation of "Energy-Based Contrastive Learning of Visual Representations", NeurIPS 2022 Oral Paper
  ☆9Updated 2 years ago
• facebookresearch / SIMAT
  codebase for the SIMAT dataset and evaluation
  ☆38Updated 2 years ago
• polo5 / FDS
  Gradient-based Hyperparameter Optimization Over Long Horizons
  ☆12Updated 3 years ago
• lucidrains / gated-state-spaces-pytorch
  Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch
  ☆95Updated last year
• ByungKwanLee / Phantom
  [Under Review] Official PyTorch implementation code for realizing the technical part of Phantom of Latent representing equipped with enla…
  ☆45Updated last month
• allenai / sso
  Repository for Skill Set Optimization
  ☆12Updated 3 months ago