DEV Community

Cover image for Youtu-Parsing-Omni: One Shared Encoder for Grounded Parsing Across Modalities
aimodels-fyi
aimodels-fyi

Posted on Originally published at aimodels.fyi

Youtu-Parsing-Omni: One Shared Encoder for Grounded Parsing Across Modalities

This is a Plain English Papers summary of a research paper called Youtu-Parsing-Omni: One Shared Encoder for Grounded Parsing Across Modalities. If you like these kinds of analyses, you can find more AI and machine-learning research on AIModels.fyi or follow us on Twitter.

Youtu-Parsing-Omni turns multimodal inputs into grounded records

Youtu-Parsing-Omni combines a shared Youtu-Omni-Encoder with OmniSchema to parse documents, natural images, audio, and video into structured records. Its outputs attach image content to bounding boxes and audio or video content to timestamps, so a downstream language model can locate the evidence behind a claim rather than repeatedly process raw media.

Diagram connecting documents, images, audio, and video to one encoder and a shared parsing schema.
The overview connects four input modalities to Youtu-Omni-Encoder and OmniSchema. It summarizes the paper’s central claim that shared perception and a unified output format can serve multiple parsing tasks.

The system covers ten tasks, including document, chart, chemical-structure, sheet-music, audio, and video parsing. A single 5B-parameter checkpoint scores 96.96 Overall on OmniDocBench v1.6 and 75.08 average on OmniParsingBench, where it leads open-weight models; the reported advantage is concentrated in audio and video....


Continue reading the full paper summary on AIModels.fyi →

Top comments (0)