Semantic segmentation of textured mosaics

EURASIP Journal on Image and Video Processing

Research

Amanda Dash¹  Alexandra Branzan Albu¹  Melissa Cote¹ 

[1] Electrical and Computer Engineering, University of Victoria, BC, Victoria, Canada;

关键词: Texture segmentation;    Semantic segmentation network;    Textures in the wild;    Visual attributes;    Deep learning;    Texture synthesis;   

DOI  :  10.1186/s13640-023-00613-0

 received in 2022-06-24, accepted in 2023-08-07,  发布年份 2023

来源: Springer

【 摘 要 】

This paper investigates deep learning (DL)-based semantic segmentation of textured mosaics. Existing popular datasets for mosaic texture segmentation, designed prior to the DL era, have several limitations: (1) training images are single-textured and thus differ from the multi-textured test images; (2) training and test textures are typically cut out from the same raw images, which may hinder model generalization; (3) each test image has its own limited set of training images, thus forcing an inefficient training of one model per test image from few data. We propose two texture segmentation datasets, based on the existing Outex and DTD datasets, that are suitable for training semantic segmentation networks and that address the above limitations: SemSegOutex focuses on materials acquired under controlled conditions, and SemSegDTD focuses on visual attributes of textures acquired in the wild. We also generate a synthetic version of SemSegOutex via texture synthesis that can be used in the same way as standard random data augmentation. Finally, we study the performance of the state-of-the-art DeepLabv3+ for textured mosaic segmentation, which is excellent for SemSegOutex and variable for SemSegDTD. Our datasets allow us to analyze results according to the type of material, visual attributes, various image acquisition artifacts, and natural versus synthetic aspects, yielding new insights into the possible usage of recent DL technologies for texture analysis.

【 授权许可】

CC BY   
© Springer Nature Switzerland AG 2023

【 图 表 】

Fig. 1

40798_2023_622_Article_IEq15.gif

40517_2023_263_Article_IEq1.gif

Fig. 10

Scheme 1

Fig. 3

40798_2023_622_Article_IEq9.gif

Fig. 2

Fig. 4

12888_2023_4959_Article_IEq3.gif

Fig. 4

12888_2023_4959_Article_IEq1.gif

Fig. 12

Fig. 1

Fig. 3

Fig. 2

Fig. 11

Fig. 3

Fig. 1

Fig. 1

12938_2023_1137_Article_IEq68.gif

Fig. 2

40517_2023_267_Article_IEq8.gif

Fig. 10

Fig. 1

Fig. 1

Fig. 1

Fig. 1

Fig. 7

Fig. 4

Fig. 3

Fig. 7

Fig. 3

Fig. 5

40798_2023_622_Article_IEq7.gif

40798_2023_622_Article_IEq6.gif

40798_2023_622_Article_IEq4.gif

Fig. 5

Fig. 9

40798_2023_622_Article_IEq1.gif

Fig. 2

41512_2023_153_Article_IEq116.gif

Fig. 1

Fig. 3

Fig. 3

Fig. 4

Fig. 1

12938_2023_1137_Article_IEq32.gif

40517_2023_267_Article_IEq6.gif

40517_2023_267_Article_IEq5.gif

40517_2023_267_Article_IEq4.gif

Fig. 2

Fig. 2

Fig. 3

Fig. 5

Fig. 8

Fig. 1

Fig. 1

Fig. 7

Fig. 1

Fig. 3

41512_2023_153_Article_IEq102.gif

41512_2023_153_Article_IEq101.gif

Fig. 2

Fig. 1

Fig. 1

Fig. 2

Fig. 4

41512_2023_153_Article_IEq93.gif

Fig. 6

Fig. 3

Fig. 2

Fig. 1

Fig. 5

Fig. 5

Fig. 4

Fig. 6

Fig. 3

41512_2023_153_Article_IEq73.gif

Fig. 1

Fig. 2

Fig. 1

Fig. 4

Fig. 14

Fig. 4

Fig. 2

Fig. 2

13570_2023_282_Article_IEq17.gif

13570_2023_282_Article_IEq6.gif

13570_2023_282_Article_IEq3.gif

13690_2023_1151_Article_IEq15.gif

Fig. 5

13690_2023_1151_Article_IEq7.gif