stan-anony / derivative_free_lora_rank

Related projects ⓘ

Alternatives and complementary repositories for derivative_free_lora_rank

• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆23Updated 4 months ago
• mjy1111 / PEAK
  The repository for our paper: Neighboring Perturbations of Knowledge Editing on Large Language Models
  ☆15Updated 6 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆44Updated last year
• locuslab / T-MARS
  Code for T-MARS data filtering
  ☆35Updated last year
• IST-DASLab / RoSA
  ☆35Updated 9 months ago
• RobertCsordas / linear_layer_as_attention
  The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …
  ☆16Updated last year
• MohammadrezaBanaei / LoRA-XS
  LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
  ☆23Updated 2 weeks ago
• VITA-Group / Data-Efficient-Scaling
  [ICML 2023] "Data Efficient Neural Scaling Law via Model Reusing" by Peihao Wang, Rameswar Panda, Zhangyang Wang
  ☆13Updated 10 months ago
• formll / resolving-scaling-law-discrepancies
  ☆15Updated 4 months ago
• yale-nlp / refdpo
  ☆15Updated 3 months ago
• mengxiayu / LLMSuperWeight
  Code for studying the super weight in LLM
  ☆16Updated last week
• twinkle0331 / LGTM
  [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…
  ☆37Updated last year
• kamanphoebe / Look-into-MoEs
  A Closer Look into Mixture-of-Experts in Large Language Models
  ☆40Updated 3 months ago
• ctlllll / reward_collapse
  ☆26Updated last year
• MJ-Bench / MJ-Bench
  Official implementation for "MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?"
  ☆38Updated this week
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆15Updated 5 months ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆39Updated 2 years ago
• DRSY / KV_Compression
  [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens
  ☆21Updated last year
• decoding-comp-trust / comp-trust
  Codebase for decoding compressed trust.
  ☆20Updated 6 months ago
• MikaStars39 / StableMask
  PyTorch implementation of StableMask (ICML'24)
  ☆12Updated 4 months ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆69Updated last month
• sail-sg / Attention-Sink
  [ATTRIB @ NeurIPS 2024] When Attention Sink Emerges in Language Models: An Empirical View
  ☆29Updated last month
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆39Updated 2 months ago
• pilancilab / Riemannian_Preconditioned_LoRA
  source code for paper "Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models"
  ☆18Updated 5 months ago
• radarFudan / Curse-of-memory
  Curse-of-memory phenomenon of RNNs in sequence modelling
  ☆19Updated this week
• maszhongming / ParaKnowTransfer
  Code for "Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective"
  ☆30Updated 6 months ago
• RAIVNLab / MatFormer-OLMo
  Code repository for the public reproduction of the language modelling experiments on "MatFormer: Nested Transformer for Elastic Inference…
  ☆18Updated last year
• pratyushmaini / llm_dataset_inference
  Official Repository for Dataset Inference for LLMs
  ☆23Updated 3 months ago
• sail-sg / MMCBench
  ☆27Updated 9 months ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 7 months ago