Alternatives and similar repositories for infini-attention:

Users that are interested in infini-attention are comparing it to the libraries listed below

• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆128Updated 8 months ago
• MBZUAI-LLM / web2code
  Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
  ☆73Updated 3 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆120Updated last month
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆140Updated 4 months ago
• Bui1dMySea / MemLong
  ☆89Updated 2 months ago
• OpenSparseLLMs / Open-Pandora
  Open-Pandora: On-the-fly Control Video Generation
  ☆32Updated 2 months ago
• RUCAIBox / Virgo
  Official code of *Virgo: A Preliminary Exploration on Reproducing o1-like MLLM*
  ☆86Updated last month
• FlagOpen / Infinity-Instruct
  ☆45Updated 8 months ago
• FudanNLPLAB / MouSi
  ☆73Updated 11 months ago
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆114Updated 4 months ago
• THUDM / LongAlign
  [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs
  ☆241Updated 2 months ago
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆78Updated 3 months ago
• Alpha-VLLM / WeMix-LLM
  ☆17Updated last year
• GAIR-NLP / ProX
  Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
  ☆212Updated this week
• 18907305772 / FuseAI
  FuseAI Project
  ☆83Updated 3 weeks ago
• jshuadvd / LongRoPE
  Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
  ☆125Updated 6 months ago
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆76Updated 11 months ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆184Updated 5 months ago
• SparksJoe / Prism
  A Framework for Decoupling and Assessing the Capabilities of VLMs
  ☆40Updated 7 months ago
• cnzzx / VSA
  Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines
  ☆114Updated 3 months ago
• RhapsodyAILab / MiniCPM-V-Embedding
  ☆27Updated 5 months ago
• LinWeizheDragon / FLMR
  The huggingface implementation of Fine-grained Late-interaction Multi-modal Retriever.
  ☆81Updated 2 weeks ago
• dwzhu-pku / PoSE
  Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)
  ☆204Updated 8 months ago
• SalesforceAIResearch / GemFilter
  ☆74Updated last month
• westlake-baichuan-mllm / bc-omni
  Baichuan-Omni: Towards Capable Open-source Omni-modal LLM 🌊
  ☆260Updated 2 weeks ago
• SparkJiao / llama-pipeline-parallel
  A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…
  ☆55Updated last year
• FreedomIntelligence / LongLLaVA
  LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture
  ☆189Updated last month
• Wangmerlyn / MCTS-GSM8k-Demo
  This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems
  ☆47Updated last month