This is a Plain English Papers summary of a research paper called LOWN Cuts Diffusion VLM Decoding FLOPs by Routing Image Access on Demand. If you like these kinds of analyses, you can find more AI and machine-learning research on AIModels.fyi or follow us on Twitter.
LOWN cuts repeated image processing in diffusion VLM decoding
LOWN (Look Only When Needed) reduces decoding cost in LLaDA-V by deciding at each denoising step whether the image must be processed. The authors find that removing visual tokens changes a token selected for unmasking at only 22.8% of steps on average, with rates ranging from 7.9% on MMBench to 43.8% on DocVQA.
The motivation is unusually concrete: visual tokens account for 89–97% of LLaDA-V’s input on the evaluated benchmarks, yet a TEXT-ONLY forward costs just 3–10% of a FULL image-conditioned forward. In paired predictions along the standard decoding trajectory, FULL and TEXT-ONLY agree on 83.4% of positions selected for unmasking, and 85.0% of final tokens had appeared as top-1 predictions before they were unmasked.
The overview contrasts decoding methods and shows how visual tokens dominate input cost. This motivates reducing repeated image-conditioned computation rather than only shortening the visual sequence....
Top comments (0)