LMM101 / Awesome-Multimodal-Next-Token-Prediction
Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey
☆209Updated this week
Alternatives and similar repositories for Awesome-Multimodal-Next-Token-Prediction:
Users that are interested in Awesome-Multimodal-Next-Token-Prediction are comparing it to the libraries listed below
- 📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.☆315Updated 2 weeks ago
- VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation☆194Updated 2 months ago
- Official implementation of the Law of Vision Representation in MLLMs☆145Updated last month
- [TMLR] Public code repo for paper "A Single Transformer for Scalable Vision-Language Modeling"☆127Updated last month
- [ECCV 2024 Oral] Code for paper: An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Langua…☆330Updated this week
- 🔥 Official impl. of "TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation".☆219Updated last week
- [NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"☆160Updated 3 months ago
- LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models☆109Updated 7 months ago
- LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture☆187Updated this week
- A curated list of awesome Multimodal studies.☆120Updated this week
- [NeurIPS'24 Spotlight] EVE: Encoder-Free Vision-Language Models☆257Updated 3 months ago
- Explore the Limits of Omni-modal Pretraining at Scale☆96Updated 4 months ago
- Harnessing 1.4M GPT4V-synthesized Data for A Lite Vision-Language Model☆251Updated 6 months ago
- Official repository of MMDU dataset☆79Updated 3 months ago
- Long Context Transfer from Language to Vision☆353Updated last month
- ☆91Updated 6 months ago
- [NeurIPS 2024] Needle In A Multimodal Haystack (MM-NIAH): A comprehensive benchmark designed to systematically evaluate the capability of…☆108Updated last month
- Reading notes about Multimodal Large Language Models, Large Language Models, and Diffusion Models☆231Updated this week
- ✨✨Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Models☆146Updated 2 weeks ago
- OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text☆298Updated last month
- A paper list of some recent works about Token Compress for Vit and VLM☆265Updated this week
- ☆33Updated last week
- Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models☆126Updated 3 weeks ago
- This repo contains the code and data for "VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks"☆107Updated last week
- The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss". A su…☆217Updated 2 months ago
- The official repository for "2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining"☆97Updated this week
- A Survey on Benchmarks of Multimodal Large Language Models☆76Updated last week
- [CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allo…☆301Updated 4 months ago
- [ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions☆186Updated 6 months ago
- LVBench: An Extreme Long Video Understanding Benchmark☆70Updated 4 months ago