Background-foreground segmentation for interior sensing in automotive industry

Journal of Mathematics in Industry

Research

Klaus Friedrichs¹  Thomas Kurbiel¹  Hanno Gottschalk²  Claudia Drygala²  Matthias Rottmann² 

[1] Aptiv Services Deutschland GmbH, Wuppertal, Germany;School of Mathematics and Science, IZMD & IMACM, University of Wuppertal, Wuppertal, Germany;

关键词: Background-foreground segmentation;    Background subtraction;    Interior sensing;    Gaussian mixture model;    Morphological snakes;    Mask R-CNN;    Augmentation;    Color spaces;    Contrast enhancement;   

DOI  :  10.1186/s13362-022-00128-9

 received in 2021-10-27, accepted in 2022-12-20,  发布年份 2022

来源: Springer

【 摘 要 】

To ensure safety in automated driving, the correct perception of the situation inside the car is as important as its environment. Thus, seat occupancy detection and classification of detected instances play an important role in interior sensing. By the knowledge of the seat occupancy status, it is possible to, e.g., automate the airbag deployment control. Furthermore, the presence of a driver, which is necessary for partially automated driving cars at the automation levels two to four can be verified. In this work, we compare different statistical methods from the field of image segmentation to approach the problem of background-foreground segmentation in camera based interior sensing. In the recent years, several methods based on different techniques have been developed and applied to images or videos from different applications. The peculiarity of the given scenarios of interior sensing is, that the foreground instances and the background both contain static as well as dynamic elements. In data considered in this work, even the camera position is not completely fixed. We review and benchmark three different methods ranging, i.e., Gaussian Mixture Models (GMM), Morphological Snakes and a deep neural network, namely a Mask R-CNN. In particular, the limitations of the classical methods, GMM and Morphological Snakes, for interior sensing are shown. Furthermore, it turns, that it is possible to overcome these limitations by deep learning, e.g. using a Mask R-CNN. Although only a small amount of ground truth data was available for training, we enabled the Mask R-CNN to produce high quality background-foreground masks via transfer learning. Moreover, we demonstrate that certain augmentation as well as pre- and post-processing methods further enhance the performance of the investigated methods.

【 授权许可】

CC BY   
© The Author(s) 2022

【 图 表 】

13731_2022_257_Article_IEq7.gif

Fig. 2

Fig. 1

13731_2022_257_Article_IEq4.gif

Fig. 3

13731_2022_257_Article_IEq2.gif

Fig. 3

Fig. 1

Fig. 6

Fig. 5

Fig. 2

Fig. 2

Fig. 5

Fig. 1

Fig. 1

Fig. 1

Fig. 1

40708_2022_178_Article_IEq64.gif

Fig. 2

Fig. 1

Fig. 1

Fig.5

Fig. 5

Fig. 4

Fig. 2

Fig. 3

Fig. 5

40708_2022_178_Article_IEq53.gif

Fig. 2

Fig. 1

Fig. 6

Fig. 4

Fig. 1

Fig. 1

Fig. 4

Fig. 1

Fig. 7

40708_2022_178_Article_IEq42.gif

Fig. 3

Fig. 6

40708_2022_178_Article_IEq39.gif

Fig. 6

Fig. 2

40708_2022_178_Article_IEq35.gif

40517_2022_243_Article_IEq20.gif

40517_2022_243_Article_IEq19.gif

Fig. 1

Fig. 4

Fig. 2

Fig. 3

40708_2022_178_Article_IEq25.gif

Fig. 2

Fig. 3

40517_2022_243_Article_IEq17.gif

40517_2022_243_Article_IEq16.gif

Fig. 2

Fig. 1

40708_2022_178_Article_IEq17.gif

Fig. 3

Fig. 5

Fig. 1

40517_2022_243_Article_IEq14.gif

Fig. 2

40517_2022_243_Article_IEq12.gif

40517_2022_243_Article_IEq11.gif

Fig. 1

12864_2022_9026_Article_IEq3.gif

Fig. 2

Fig. 4

Fig. 1

Fig. 2

Fig. 3

40517_2022_243_Article_IEq10.gif

Fig. 4

Fig. 9

Fig. 3

Fig. 3

Fig. 4

Fig. 4

40517_2022_243_Article_IEq8.gif

Fig. 2

Fig. 3

Fig. 1

Fig. 3

Fig. 2

Fig. 3

40517_2022_243_Article_IEq5.gif

40517_2022_243_Article_IEq4.gif

Fig. 1

Fig. 7

Fig. 1

Fig. 2

Fig. 4

Fig. 4

40708_2022_178_Article_IEq7.gif

40517_2022_243_Article_IEq3.gif

Scheme 1

Fig. 2

Fig. 2

Fig. 1

Fig. 1

Fig. 2

Fig. 7

Fig. 2

40517_2022_243_Article_IEq2.gif

40517_2022_243_Article_IEq1.gif

Fig. 1

Fig. 3

Fig. 4

Fig. 6

Fig. 4

12951_2022_1737_Article_IEq1.gif

Fig. 8

Fig. 2

Fig. 3

Fig. 4

Fig. 2

Fig. 4

Fig. 1

Fig. 3

Fig. 2

Fig. 1

Fig. 1

Fig. 2

Fig. 1

Fig. 2

Fig. 1

Fig. 4

Fig. 6

Fig. 1

Fig. 4

Fig. 4

Fig. 6

Fig. 3

Fig. 3

Fig. 1

Fig. 5

Fig. 1

Fig. 5

Fig. 3

Fig. 5

Fig. 1

Fig. 2

Fig. 4

Fig. 1

Fig. 3

Fig. 4

Fig.1

Fig. 1

Fig. 1

Fig. 3

Fig. 4

Fig. 1 (abstract P46).

Fig. 3

Fig. 1

Fig. 1

Fig. 4

Fig. 2

40560_2022_645_Article_IEq3.gif

40560_2022_645_Article_IEq2.gif

Scheme 1

Fig. 2

Fig. 1

Fig. 3

Fig. 1

Fig. 1

Fig. 13

Fig. 2

Fig. 3

Fig. 7

Fig. 1

Fig. 6

Fig. 6

Fig. 11

Fig. 2

Fig. 9

Fig. 1

Fig. 4

12936_2022_4386_Article_IEq183.gif

Fig. 10

Fig. 7

Fig. 3

12936_2022_4386_Article_IEq179.gif

Fig. 4

Fig. 2

Fig. 2

Fig. 2

Fig. 1

40644_2022_517_Article_IEq1.gif

Fig. 1

Fig. 1

12936_2022_4386_Article_IEq169.gif

Fig. 6

Fig. 1

Fig. 9

Fig. 5

Fig. 6

Fig. 1

Fig. 1

Fig. 1

Fig. 4

Fig. 8

Fig. 3

Fig. 7

Fig. 3

12936_2022_4386_Article_IEq146.gif

Fig. 1

Fig. 2

Fig. 6

12936_2022_4386_Article_IEq142.gif

Fig. 1

Fig. 1

Fig. 1

Fig. 2