How well do Text-to-Image models actually follow complex instructions? Imag-Eval provides interpretable, skill-based evaluation of T2I instruction following, without error propagation. 📄 Accepted at EMNLP 2026 | ⭐ Check it out & star the repo!
Justsecret123
/
Imag-Eval
Imag-Eval [EMNLP 2026] is a skill-based evaluation framework for Text-to-Image (T2I) models that measures instruction-following capabilities across compositional visual reasoning skills, while explicitly controlling for prompt complexity and minimizing error propagation during evaluation.
Imag-Eval [EMNLP 2026]
Official repository for the paper "Imag-Eval A language-grounded framework for interpretable Text-to-Image instruction following evaluation". (SEROUIS et al., EMNLP 2026)
TL;DR: Imag-Eval is a skill-based evaluation framework for Text-to-Image (T2I) models that measures instruction-following capabilities across compositional visual reasoning skills, while explicitly controlling for prompt complexity and minimizing error propagation during evaluation; we are rying to shift evaluation paradigms towards more controlled increases in complexity.
To ensure leaderboard integrity and reproducibility, submissions must include the generated images, generation seed, the method used for annotation, and all relevant inference parameters. Reported results will be independently verified, and entries whose reproduced results closely match the submitted scores will be added to the leaderboard.
🚀 Recent News
- [Sept 2026] 🎉 The official leaderboard is now available.
- [Aug 2026] 🎉 IMAG-EVAL has been accepted to EMNLP 2026.
- [Aug 2026] 📊 Released the first version of the IMAG-EVAL benchmark…
Top comments (0)