mit-han-lab / vila-u

Related projects ⓘ

Alternatives and complementary repositories for vila-u

• AILab-CVC / VL-GPT
  VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation
  ☆84Updated 2 months ago
• rongyaofang / PUMA
  Empowering Unified MLLM with Multi-granular Visual Generation
  ☆104Updated 3 weeks ago
• MonoFormer / MonoFormer
  The official implementation for "MonoFormer: One Transformer for Both Diffusion and Autoregression"
  ☆76Updated last month
• AdaCache-DiT / AdaCache
  Adaptive Caching for Faster Video Generation with Diffusion Transformers
  ☆79Updated last week
• bronyayang / Law_of_Vision_Representation_in_MLLMs
  Official implementation of the Law of Vision Representation in MLLMs
  ☆128Updated 2 months ago
• yfzhang114 / SliME
  ✨✨Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Models
  ☆137Updated this week
• ChaofanTao / Autoregressive-Models-in-Vision-Survey
  The paper collections for the autoregressive models in vision.
  ☆95Updated last week
• Cooperx521 / PyramidDrop
  The official code of the paper "PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction".
  ☆41Updated last week
• baaivision / EVE
  [NeurIPS'24 Spotlight] EVE: Encoder-Free Vision-Language Models
  ☆227Updated last month
• mutonix / Vript
  ☆126Updated last week
• FoundationVision / vaex
  🔥stable, simple, state-of-the-art VQVAE toolkit & cookbook
  ☆40Updated 4 months ago
• MMStar-Benchmark / MMStar
  [NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"
  ☆147Updated last month
• NJU-PCALab / OpenVid-1M
  ☆189Updated 3 months ago
• invictus717 / MiCo
  Explore the Limits of Omni-modal Pretraining at Scale
  ☆86Updated 2 months ago
• lxa9867 / Awesome-Autoregressive-Visual-Generation
  This is a repo to track the latest autoregressive visual generation papers.
  ☆42Updated last month
• rese1f / aurora
  🔥 Aurora Series: A more efficient multimodal large language model series for video.
  ☆41Updated 2 weeks ago
• mit-han-lab / hart
  HART: Efficient Visual Generation with Hybrid Autoregressive Transformer
  ☆324Updated 3 weeks ago
• Beckschen / LLaVolta
  [NeurIPS 2024] Efficient Multi-modal Models via Stage-wise Visual Context Compression
  ☆38Updated 3 months ago
• LeapLabTHU / AdaNAT
  [ECCV 2024] AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation
  ☆31Updated 2 months ago
• DAMO-NLP-SG / Inf-CLIP
  The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss". A su…
  ☆164Updated last week
• Liuziyu77 / MIA-DPO
  Official implement of MIA-DPO
  ☆32Updated last week
• FreedomIntelligence / LongLLaVA
  LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture
  ☆178Updated last month
• showlab / Awesome-Unified-Multimodal-Models
  📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.
  ☆205Updated this week
• llyx97 / TempCompass
  [ACL 2024 Findings] "TempCompass: Do Video LLMs Really Understand Videos?", Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, …
  ☆83Updated 3 weeks ago
• dvlab-research / Prompt-Highlighter
  [CVPR 2024] Prompt Highlighter: Interactive Control for Multi-Modal LLMs
  ☆132Updated 3 months ago
• EvolvingLMMs-Lab / LongVA
  Long Context Transfer from Language to Vision
  ☆328Updated 2 weeks ago
• UX-Decoder / FIND
  [NeurIPS 2024] Official implementation of the paper "Interfacing Foundation Models' Embeddings"
  ☆110Updated 2 months ago
• KangarooGroup / Kangaroo
  official impelmentation of Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input
  ☆54Updated 2 months ago
• daixiangzi / VAR-CLIP
  Implements VAR+CLIP for image generation
  ☆78Updated 3 months ago
• horseee / learning-to-cache
  [NeurIPS 2024] Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching
  ☆71Updated 3 months ago