Abstract

We introduce HEaD (Hallucination Early Detection), a novel paradigm designed to swiftly detect incorrect generations at the beginning of the diffusion process. Diffusion models struggle when generating complex combinations of multiple objects, often requiring multiple iterations with different seeds, wasting time and increasing energy consumption. Our HEaD pipeline combines cross-attention maps with a new indicator, the Predicted Final Image, to forecast the final outcome using information available in early generation stages. Results show HEaD can save up to 12% of generation time in two-object scenarios, highlighting the importance of early detection mechanisms in improving efficiency for complex generative tasks.

Key Contributions

1. Novel HEaD paradigm for early hallucination detection in diffusion models
2. Combination of cross-attention maps with Predicted Final Image indicator
3. Quantitative analysis showing up to 12% time savings for multi-object generation
4. Framework for improving efficiency and reducing energy consumption in complex generative tasks