codefuse-ai / Collinear-Constrained-Attention

Related projects: ⓘ

• SparkJiao / llama-pipeline-parallel
  A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…
  ☆45Updated last year
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆121Updated 3 months ago
• princeton-nlp / CEPE
  [ACL 2024] Long-Context Language Modeling with Parallel Encodings
  ☆133Updated 3 months ago
• tianyi-lab / Superfiltering
  [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
  ☆101Updated 2 weeks ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆114Updated 2 months ago
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆72Updated 6 months ago
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆59Updated 9 months ago
• FlagOpen / Infinity-Instruct
  ☆32Updated 3 months ago
• zhaochenyang20 / Prompt2Model-Self-Guide
  SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper
  ☆21Updated 3 months ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆68Updated 11 months ago
• Bui1dMySea / MemLong
  ☆57Updated 3 weeks ago
• TemporaryLoRA / Temp-LoRA
  ☆82Updated 5 months ago
• nishiwen1214 / Benchmark-leakage-detection
  Official completion of “Training on the Benchmark Is Not All You Need”.
  ☆18Updated last week
• sail-sg / sdft
  [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
  ☆81Updated this week
• THUDM / NaturalCodeBench
  ☆52Updated 2 months ago
• liuqidong07 / MOELoRA-peft
  [SIGIR'24] The official implementation code of MOELoRA.
  ☆113Updated 2 months ago
• yegcjs / mixinglaws
  ☆87Updated 4 months ago
• shizhediao / Post-Training-Data-Flywheel
  We aim to provide the best references to search, select, and synthesize high-quality and large-quantity data for post-training your LLMs.
  ☆38Updated 2 weeks ago
• alibaba / ChatLearn
  ☆105Updated last week
• zexuanqiu / CLongEval
  CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models
  ☆37Updated 6 months ago
• cofe-ai / Mu-scaling
  Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales
  ☆30Updated last year
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting
  ☆60Updated 6 months ago
• THUDM / ChatGLM-Math
  ☆75Updated 5 months ago
• sail-sg / regmix
  🧬 RegMix: Data Mixture as Regression for Language Model Pre-training
  ☆79Updated this week
• gpt4life / alpagasus
  Unofficial implementation of AlpaGasus
  ☆83Updated 11 months ago
• ZhenweiAn / Dynamic_MoE
  Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"
  ☆17Updated last month
• fxmeng / mixtral_spliter
  Converting Mixtral-8x7B to Mixtral-[1~7]x7B
  ☆20Updated 6 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆32Updated 8 months ago
• wwxu21 / CUT
  Source code of "Reasons to Reject? Aligning Language Models with Judgments"
  ☆54Updated 6 months ago
• THUDM / LongAlign
  LongAlign: A Recipe for Long Context Alignment Encompassing Data, Training, and Evaluation
  ☆194Updated 4 months ago