ASMCC-Diff: Arbitrary Size Multi-Condition Controllable Chinese Landscape Painting Generation with Diffusion Models

Résumé fourni par la source

Thanks to the emergence of generative models, Chinese Landscape Painting Generation (CLPG) has garnered increasing attention. However, existing works are primarily limited to relying on text control conditions, lacking more fine-grained control over spatial layout and style. Additionally, they are limited to a fixed size and aspect ratios. But different landscape scenes require different sizes to appear more balanced and natural. Thus, a question arises: Is it possible to design a model that can be controlled by multiple conditions (text, style and sketches) while generating images with various sizes? In this paper, we explore this issue and propose ASMCC-Diff. Specifically, it consists of two modules, i.e. multi-condition controlled image generation module and arbitrary size up-scaling module. The critical insights of multi-condition controlled image generation module are to embed multiple conditions with distinct priorities. The sketch condition serves as the primary flow, guiding the overall structure, while the text and style conditions act as auxiliary components, injected into the diffusion model via a cross-attention module. Additionally, to avoid conflicts between the semantics of style images and text, we use Q-former to separate the semantic and stylistic information of the reference image. For the arbitrary size upscaling module, we first truncate the generation process, up-sample the image to the specified size, and then continue the generation. Furthermore, we introduce a new Chinese landscape painting database that supports multiple conditions, facilitating further research. Experimental results demonstrate the superior performance of our proposed model. The code and dataset will be released.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.