The latest articles on DEV Community by Secret123 (@justsecret123). https://dev.to/justsecret123 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4110061%2Fbe21d514-37c4-43aa-8d14-8e63a7d4f743.png https://dev.to/justsecret123 en Secret123 Fri, 04 Sep 2026 15:46:30 +0000 https://dev.to/justsecret123/imag-eval-5dn7 https://dev.to/justsecret123/imag-eval-5dn7 <p>How well do Text-to-Image models actually follow complex instructions? Imag-Eval provides interpretable, skill-based evaluation of T2I instruction following, without error propagation. 📄 Accepted at EMNLP 2026 | ⭐ Check it out &amp; star the repo!</p> <div class="ltag-github-readme-tag"> <div class="readme-overview"> <h2> <img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"> <a href="https://github.com/Justsecret123" rel="noopener noreferrer"> Justsecret123 </a> / <a href="https://github.com/Justsecret123/Imag-Eval" rel="noopener noreferrer"> Imag-Eval </a> </h2> <h3> Imag-Eval [EMNLP 2026] is a skill-based evaluation framework for Text-to-Image (T2I) models that measures instruction-following capabilities across compositional visual reasoning skills, while explicitly controlling for prompt complexity and minimizing error propagation during evaluation. </h3> </div> <div class="ltag-github-body"> <div id="readme" class="md"><p><a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/f83d609947fc061dd3877bbb6e91e6380ea4fbd79a79830714f9c67eccd442d7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f507974686f6e2d332e31312e342d627269676874677265656e3f7374796c653d666f722d7468652d6261646765266c6f676f3d507974686f6e"><img src="https://camo.githubusercontent.com/f83d609947fc061dd3877bbb6e91e6380ea4fbd79a79830714f9c67eccd442d7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f507974686f6e2d332e31312e342d627269676874677265656e3f7374796c653d666f722d7468652d6261646765266c6f676f3d507974686f6e" alt="Static Badge"></a> <a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/aeb030be2305e22c62bc9ff4996d9f507a0f7cdd6ecfc15d5438ce468e176d6a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5079546f7263682d322e31302e3025324263753132382d627269676874677265656e3f7374796c653d666f722d7468652d6261646765266c6f676f3d5079746f726368"><img src="https://camo.githubusercontent.com/aeb030be2305e22c62bc9ff4996d9f507a0f7cdd6ecfc15d5438ce468e176d6a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5079546f7263682d322e31302e3025324263753132382d627269676874677265656e3f7374796c653d666f722d7468652d6261646765266c6f676f3d5079746f726368" alt="Static Badge"></a></p> <div> <div class="markdown-heading"> <h1 class="heading-element">Imag-Eval [EMNLP 2026]</h1> </div> <p><strong>Official repository for the paper "<a href="https://arxiv.org/abs/2608.29210" rel="nofollow noopener noreferrer">Imag-Eval A language-grounded framework for interpretable Text-to-Image instruction following evaluation</a>". (SEROUIS et al., EMNLP 2026)</strong></p> <p><strong>TL;DR:</strong> Imag-Eval is a skill-based evaluation framework for Text-to-Image (T2I) models that measures instruction-following capabilities across compositional visual reasoning skills, while explicitly controlling for prompt complexity and minimizing error propagation during evaluation; we are rying to shift evaluation paradigms towards more controlled increases in complexity.</p> <p><a href="https://arxiv.org/pdf/2608.29210" rel="nofollow noopener noreferrer"><img src="https://camo.githubusercontent.com/bde6aaa49e40e0735dccae0b6d51d6b627ca2569a3fc05df0185eb9798fa01a7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50617065722d61725869762d6233316231623f7374796c653d666f722d7468652d6261646765266c6f676f3d6172786976266c6f676f436f6c6f723d7768697465" alt="Paper"></a> <a href="https://huggingface.co/datasets/JustSecret/Imag-Eval" rel="nofollow noopener noreferrer"><img src="https://camo.githubusercontent.com/9ef6d1e7ed512af1df5555fd7bdd1dc66162cc675c861515148306482a985a49/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f446174617365742d48756767696e67466163652d4646443231453f7374796c653d666f722d7468652d6261646765266c6f676f3d68756767696e6766616365266c6f676f436f6c6f723d626c61636b" alt="Dataset"></a> <a href="https://justsecret123.github.io/imag-eval-leaderboard.io" rel="nofollow noopener noreferrer"><img src="https://camo.githubusercontent.com/e1172b4c110a501fe5c9bdc3b93b46bebb2962d3a8ad61eb90211475259b2de5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c6561646572626f6172642d4c6976652d3633363666313f7374796c653d666f722d7468652d6261646765266c6f676f3d6769746875627061676573266c6f676f436f6c6f723d7768697465" alt="Leaderboard"></a></p> </div> <blockquote> <p>To ensure leaderboard integrity and reproducibility, submissions must include the generated images, generation seed, the method used for annotation, and all relevant inference parameters. Reported results will be independently verified, and entries whose reproduced results closely match the submitted scores will be added to the leaderboard.</p> </blockquote> <div class="markdown-heading"> <h2 class="heading-element">🚀 Recent News</h2> </div> <ul> <li> <strong>[Sept 2026]</strong> 🎉 <a href="https://justsecret123.github.io/imag-eval-leaderboard.io" rel="nofollow noopener noreferrer">The official leaderboard</a> is now available.</li> <li> <strong>[Aug 2026]</strong> 🎉 IMAG-EVAL has been accepted to <strong>EMNLP 2026</strong>.</li> <li> <strong>[Aug 2026]</strong> 📊 Released the first version of the IMAG-EVAL benchmark…</li> </ul></div> </div> <div class="gh-btn-container"><a class="gh-btn" href="https://github.com/Justsecret123/Imag-Eval" rel="noopener noreferrer">View on GitHub</a></div> </div> ai machinelearning deeplearning computervision