Alternatives and similar repositories for language_modeling_is_compression:

Users that are interested in language_modeling_is_compression are comparing it to the libraries listed below

• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]
  ☆132Updated 7 months ago
• sustcsonglin / linear-attention-and-beyond-slides
  ☆69Updated 2 months ago
• HazyResearch / based
  Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
  ☆232Updated 2 months ago
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆212Updated last year
• ScalingIntelligence / large_language_monkeys
  ☆89Updated 6 months ago
• NVIDIA / ngpt
  Normalized Transformer (nGPT)
  ☆168Updated 5 months ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆156Updated last year
• JacobPfau / fillerTokens
  ☆60Updated 11 months ago
• agentica-project / verl-pipeline
  Async pipelined version of Verl
  ☆60Updated 2 weeks ago
• TIGER-AI-Lab / LongICLBench
  Code and Data for "Long-context LLMs Struggle with Long In-context Learning" [TMLR2025]
  ☆105Updated 2 months ago
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆107Updated last week
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆81Updated 10 months ago
• princeton-nlp / ProLong
  Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"
  ☆175Updated last month
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆73Updated last year
• mengxiayu / LLMSuperWeight
  Code for studying the super weight in LLM
  ☆98Updated 4 months ago
• gregorbachmann / Next-Token-Failures
  ☆83Updated last year
• jxiw / MambaInLlama
  [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models
  ☆214Updated last week
• princeton-nlp / HELMET
  The HELMET Benchmark
  ☆135Updated last week
• hughbzhang / o1_inference_scaling_laws
  Replicating O1 inference-time scaling laws
  ☆83Updated 4 months ago
• McGill-NLP / VinePPO
  Code for the paper "VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment"
  ☆151Updated 5 months ago
• nightdessert / Retrieval_Head
  open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality
  ☆185Updated 8 months ago
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆96Updated last year
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆121Updated 3 months ago
• llm-random / llm-random
  ☆185Updated this week
• berlino / gated_linear_attention
  ☆102Updated last year
• RobertCsordas / moeut
  ☆78Updated 8 months ago
• XunhaoLai / native-sparse-attention-triton
  Efficient triton implementation of Native Sparse Attention.
  ☆139Updated 2 weeks ago
• ryoungj / ObsScaling
  [NeurIPS'24 Spotlight] Observational Scaling Laws
  ☆54Updated 6 months ago
• vwxyzjn / summarize_from_feedback_details
  ☆137Updated 5 months ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆82Updated 6 months ago