Metamorph Multimodal Understanding And Generation Via Instruction Tuning

Yann LeCun

paper · 2024-12-18

Recommended

“In this work, we propose Visual-Predictive Instruction Tuning (VPiT) - a simple and effective extension to visual instruction tuning that enables a pretrained LLM to quickly morph into an unified autoregressive model capable of generating both text and visual tokens.”