This is a Plain English Papers summary of a research paper called Youtu-Parsing-Omni: One Shared Encoder for Grounded Parsing Across Modalities. If you like these kinds of analyses, you can find more AI and machine-learning research on AIModels.fyi or follow us on Twitter.
Youtu-Parsing-Omni turns multimodal inputs into grounded records
Youtu-Parsing-Omni combines a shared Youtu-Omni-Encoder with OmniSchema to parse documents, natural images, audio, and video into structured records. Its outputs attach image content to bounding boxes and audio or video content to timestamps, so a downstream language model can locate the evidence behind a claim rather than repeatedly process raw media.

The overview connects four input modalities to Youtu-Omni-Encoder and OmniSchema. It summarizes the paper’s central claim that shared perception and a unified output format can serve multiple parsing tasks.
The system covers ten tasks, including document, chart, chemical-structure, sheet-music, audio, and video parsing. A single 5B-parameter checkpoint scores 96.96 Overall on OmniDocBench v1.6 and 75.08 average on OmniParsingBench, where it leads open-weight models; the reported advantage is concentrated in audio and video....
Top comments (0)