Alternatives and similar repositories for language_modeling_is_compression:

Users that are interested in language_modeling_is_compression are comparing it to the libraries listed below

• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]
  ☆130Updated 4 months ago
• princeton-nlp / ProLong
  Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"
  ☆153Updated 2 months ago
• ScalingIntelligence / large_language_monkeys
  ☆82Updated 4 months ago
• princeton-nlp / HELMET
  The HELMET Benchmark
  ☆114Updated last week
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆96Updated 10 months ago
• nightdessert / Retrieval_Head
  open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality
  ☆172Updated 6 months ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆43Updated last week
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆213Updated last year
• DRSY / EasyKV
  Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
  ☆60Updated last year
• JacobPfau / fillerTokens
  ☆58Updated 9 months ago
• teelinsan / parallel-decoding
  Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"
  ☆114Updated 11 months ago
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆68Updated 10 months ago
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆196Updated 2 months ago
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆149Updated last month
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆76Updated 4 months ago
• thunlp / Modularity-Analysis
  Repo for ACL2023 Findings paper "Emergent Modularity in Pre-trained Transformers"
  ☆21Updated last year
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆81Updated 8 months ago
• hughbzhang / o1_inference_scaling_laws
  Replicating O1 inference-time scaling laws
  ☆82Updated 2 months ago
• whyNLP / LCKV
  Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…
  ☆147Updated 3 weeks ago
• thu-ml / ReMoE
  Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆55Updated last month
• thu-wyz / inference_scaling
  ☆50Updated 2 months ago
• PRIME-RL / ImplicitPRM
  Repo of paper "Free Process Rewards without Process Labels"
  ☆118Updated last month
• HazyResearch / based
  Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
  ☆220Updated 2 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆42Updated last month
• HanGuo97 / lq-lora
  ☆125Updated last year
• gregorbachmann / Next-Token-Failures
  ☆80Updated 11 months ago
• McGill-NLP / VinePPO
  Code for the paper "VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment"
  ☆120Updated 3 months ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆140Updated 11 months ago
• berlino / gated_linear_attention
  ☆99Updated 11 months ago