Co-occurrence based texture synthesis

Computational Visual Media

Hadar Averbuch-Elor¹  Anna Darzi²  Ashutosh Taklikar²  Shai Avidan²  Itai Lang² 

[1] Cornell-Tech, Cornell University, 10044, NYC, NY, USA;Tel Aviv University, 6997801, Tel Aviv, Israel;

关键词: co-occurrence;    texture synthesis;    deep learning;    generative adversarial networks (GANs);   

DOI  :  10.1007/s41095-021-0243-7

来源: Springer

【 摘 要 】

As image generation techniques mature, there is a growing interest in explainable representations that are easy to understand and intuitive to manipulate. In this work, we turn to co-occurrence statistics, which have long been used for texture analysis, to learn a controllable texture synthesis model. We propose a fully convolutional generative adversarial network, conditioned locally on co-occurrence statistics, to generate arbitrarily large images while having local, interpretable control over texture appearance. To encourage fidelity to the input condition, we introduce a novel differentiable co-occurrence loss that is integrated seamlessly into our framework in an end-to-end fashion. We demonstrate that our solution offers a stable, intuitive, and interpretable latent representation for texture synthesis, which can be used to generate smooth texture morphs between different textures. We further show an interactive texture tool that allows a user to adjust local characteristics of the synthesized texture by directly using the co-occurrence values.

【 授权许可】

CC BY