An end-to-end text spotter with text relation networks

Cybersecurity

Gang Li¹  Chao Liu²  Weiqing Huang²  Min Li²  Baole Wei³  Boquan Li³  Min Yu³  Jianguo Jiang³ 

[1] Centre for Cyber Security Research and Innovation, Deakin University, Geelong, Australia;Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China;Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China;School of Cyber Security, University of Chinese Academy of Science, Beijing, China;

关键词: Scene text spotting;    Graph convolutional network;    Visual reasoning;   

DOI  :  10.1186/s42400-021-00073-x

来源: Springer

【 摘 要 】

Reading text in images automatically has become an attractive research topic in computer vision. Specifically, end-to-end spotting of scene text has attracted significant research attention, and relatively ideal accuracy has been achieved on several datasets. However, most of the existing works overlooked the semantic connection between the scene text instances, and had limitations in situations such as occlusion, blurring, and unseen characters, which result in some semantic information lost in the text regions. The relevance between texts generally lies in the scene images. From the perspective of cognitive psychology, humans often combine the nearby easy-to-recognize texts to infer the unidentifiable text. In this paper, we propose a novel graph-based method for intermediate semantic features enhancement, called Text Relation Networks. Specifically, we model the co-occurrence relationship of scene texts as a graph. The nodes in the graph represent the text instances in a scene image, and the corresponding semantic features are defined as representations of the nodes. The relative positions between text instances are measured as the weights of edges in the established graph. Then, a convolution operation is performed on the graph to aggregate semantic information and enhance the intermediate features corresponding to text instances. We evaluate the proposed method through comprehensive experiments on several mainstream benchmarks, and get highly competitive results. For example, on the SCUT-CTW1500, our method surpasses the previous top works by 2.1% on the word spotting task.

【 授权许可】

CC BY