Alternatives and similar repositories for qformer

Users that are interested in qformer are comparing it to the libraries listed below

• 360CVGroup / Inner-Adaptor-Architecture
  LMM solved catastrophic forgetting, AAAI2025
  ☆44Updated 3 months ago
• kyegomez / Mirasol
  Pytorch Implementation of the Model from "MIRASOL3B: A MULTIMODAL AUTOREGRESSIVE MODEL FOR TIME-ALIGNED AND CONTEXTUAL MODALITIES"
  ☆26Updated 5 months ago
• lzw-lzw / UnifiedMLLM
  UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model
  ☆22Updated 11 months ago
• bojone / FSQ
  Keras implement of Finite Scalar Quantization
  ☆77Updated last year
• multimodal-art-projection / OmniBench
  A project for tri-modal LLM benchmarking and instruction tuning.
  ☆40Updated 3 months ago
• lucasjinreal / LLaVA-Magvit2
  LLaVA combines with Magvit Image tokenizer, training MLLM without an Vision Encoder. Unifying image understanding and generation.
  ☆37Updated last year
• zehanwang01 / OmniBind
  ☆32Updated 3 months ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆87Updated last year
• amazon-science / masked-diffusion-lm
  Official implementation for the paper "A Cheaper and Better Diffusion Language Model with Soft-Masked Noise"
  ☆58Updated last year
• OpenNLPLab / Tnn
  [ICLR 2023] Official implementation of Transnormer in our ICLR 2023 paper - Toeplitz Neural Network for Sequence Modeling
  ☆79Updated last year
• JaaackHongggg / WorldSense
  WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs
  ☆26Updated 2 months ago
• RainBowLuoCS / OpenOmni
  OpenOmni: Official implementation of Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Rea…
  ☆89Updated 2 weeks ago
• nbasyl / DoRA
  Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
  ☆124Updated last year
• Optimization-AI / FastCLIP
  Distributed Optimization Infra for learning CLIP models
  ☆26Updated 9 months ago
• MIO-Team / MIO
  MIO: A Foundation Model on Multimodal Tokens
  ☆27Updated 7 months ago
• kyegomez / TTL
  Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"
  ☆25Updated 2 weeks ago
• AILab-CVC / M2PT
  [CVPR'24] Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities
  ☆99Updated last year
• lucasjinreal / ImageTokenizer
  imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…
  ☆34Updated last year
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆85Updated 8 months ago
• zh460045050 / VQGAN-LC
  ☆129Updated last year
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆52Updated 5 months ago
• MCR-PEFT / Ex-MCR
  ☆45Updated last month
• lucidrains / multimodal-dit-pytorch
  Implementation of a multimodal diffusion transformer in Pytorch
  ☆102Updated last year
• csarron / PuMer
  [ACL 2023] PuMer: Pruning and Merging Tokens for Efficient Vision Language Models
  ☆32Updated 9 months ago
• transformer-vq / transformer_vq
  ☆195Updated last year
• invictus717 / MiCo
  [ICCV'25] Explore the Limits of Omni-modal Pretraining at Scale
  ☆105Updated 10 months ago
• kyegomez / PaLM2-VAdapter
  Implementation of "PaLM2-VAdapter:" from the multi-modal model paper: "PaLM2-VAdapter: Progressively Aligned Language Model Makes a Stron…
  ☆16Updated 8 months ago
• lucidrains / agent-attention-pytorch
  Implementation of Agent Attention in Pytorch
  ☆90Updated last year
• kyegomez / MultiModalCrossAttn
  The open source implementation of the cross attention mechanism from the paper: "JOINTLY TRAINING LARGE AUTOREGRESSIVE MULTIMODAL MODELS"
  ☆32Updated last year
• jadehaus / preference-flow-matching
  Official code implementation for the work Preference Alignment with Flow Matching (NeurIPS 2024)
  ☆56Updated 8 months ago