Abstract

We introduce ViCE (Visual Concept Evaluation), a novel automated method to assess consistency between generated/edited images and their corresponding prompts/instructions, inspired by human cognitive processes. Despite advances in image generation quality, no methodical frameworks exist to quantitatively measure content quality and prompt adherence. ViCE combines Large Language Models (LLMs) and Visual Question Answering (VQA) in a unified pipeline that outlines visual concepts, formulates verification questions, investigates the image through Q&A, and scores the results, offering a promising approach to automatic evaluation for increasingly sophisticated generation tasks.

Key Contributions

1. Novel evaluation framework based on human cognitive processes for image generation assessment
2. Combination of LLMs and VQA in a unified evaluation pipeline
3. Systematic approach to quantify consistency between generated images and text prompts
4. Preliminary validation showing promising results for automatic image quality evaluation