Structural and Relational Reasoning with Multi-Scale Context for Semantic Segmentation

EasyChair Preprint no. 2547

Abstract

It is important for semantic segmentation to learn two types of context information. One is global context information for understanding objects, relations between objects, and scenes in input images. The other is multi-scale context information for adapting to changes in the scale and shape of objects. In this research, we tackle the problem of learning to extract them for semantic segmentation. To achieve this, we propose a novel unit that learns to perform structural and relational reasoning by selecting the multi-scale context. The multi-scale context is extracted from receptive fields of different sizes of the backbone network and then is implicitly utilized to improve the global context obtained by GloRe. By the proposed unit, our model allows us to perform structural and relational reasoning for semantic segmentation in complex scenes. We conduct experiments on Cityscapes. In particular, our model achieves the mean IoU score of 73.6, which is 1.1% higher than GloRe. Then, by comparing the prediction between the proposed method and GloRe unit, we confirmed the effect of incorporating two types of context information.

Keyphrases: Global Reasoning, graph convolution, semantic segmentation

@Booklet{EasyChair:2547,
  author = {Yukihiro Domae and Hiroaki Aizawa and Kunihito Kato},
  title = {Structural and Relational Reasoning with Multi-Scale Context for Semantic Segmentation},
  howpublished = {EasyChair Preprint no. 2547},

  year = {EasyChair, 2020}}