kohjingyu / fromage
🧀 Code and models for the ICML 2023 paper "Grounding Language Models to Images for Multimodal Inputs and Outputs".
☆478Updated last year
Related projects ⓘ
Alternatives and complementary repositories for fromage
- 🐟 Code and models for the NeurIPS 2023 paper "Generating Images with Multimodal Language Models".☆433Updated 10 months ago
- DataComp: In search of the next generation of multimodal datasets☆658Updated 10 months ago
- Implementation of the deepmind Flamingo vision-language model, based on Hugging Face language models and ready for training☆165Updated last year
- Official Repository of ChatCaptioner☆452Updated last year
- Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch☆1,217Updated 2 years ago
- [NeurIPS 2023] This repository includes the official implementation of our paper "An Inverse Scaling Law for CLIP Training"☆298Updated 5 months ago
- ☆573Updated 9 months ago
- GIT: A Generative Image-to-text Transformer for Vision and Language☆550Updated 11 months ago
- This is the official repository for the LENS (Large Language Models Enhanced to See) system.☆351Updated 11 months ago
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities (ICML 2024)☆266Updated 2 weeks ago
- Research Trends in LLM-guided Multimodal Learning.☆355Updated last year
- Language Models Can See: Plugging Visual Controls in Text Generation☆254Updated 2 years ago
- Open reproduction of MUSE for fast text2image generation.☆332Updated 5 months ago
- CLIP-like model evaluation☆618Updated 3 months ago
- MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.☆907Updated 5 months ago
- Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.☆368Updated last year
- Code/Data for the paper: "LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding"☆258Updated 5 months ago
- Aligning LMMs with Factually Augmented RLHF☆323Updated last year
- (CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.☆315Updated 4 months ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆435Updated 7 months ago
- ☆219Updated 11 months ago
- Official implementation of SEED-LLaMA (ICLR 2024).☆583Updated 2 months ago
- Code release for "Learning Video Representations from Large Language Models"☆492Updated last year
- [ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning☆257Updated 8 months ago
- Large-scale text-video dataset. 10 million captioned short videos.☆605Updated 3 months ago
- An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal …☆363Updated 11 months ago
- LaVIT: Empower the Large Language Model to Understand and Generate Visual Content☆535Updated last month
- [NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"☆509Updated 9 months ago
- MMICL, a state-of-the-art VLM with the in context learning ability from ICL, PKU☆334Updated 11 months ago
- Get hundred of million of image+url from the crawling at home dataset and preprocess them☆207Updated 5 months ago