Title: Graph-FCN for image semantic segmentation

论文网址：https://arxiv.org/pdf/2001.00335.pdf

作者：Yi Lu1, Yaran Chen 1, Dongbin Zhao1, and Jianxin Chen

发表单位：Chinese Academy of Sciences

发表于：ISNN 2019

关键词：GCN，image segmentation

一句话总结：将图像分割问题转化为graph node分类问题，将FCN学习到的feature map作为GCN输入的node的feature vector；

0. Abstract

任务：语义分割

提出问题：the local location information is usually ignored in the high-level feature extraction；

解决方法：

• we propose a graph model initialized by a fully convolutional network (FCN) named Graph-FCN for image semantic segmentation.
• Firstly, the image grid data is extended to graph structure data by a convolutional network, whichtransforms the semantic segmentation problem into a graph node classification problem.
• Then we apply graph convolutional network to solve this graph node classification problem. As far as we know, it is the first time that we apply the graph convolutional network in image semantic segmentation.
• Our method achieves competitive performance in mean intersection over union (mIOU) on theVOC dataset(about 1.34% improvement), compared to the original FCN model.

1. Introduction

介绍任务：介绍了语义分割任务和相关的DL框架；

提出问题：高级特征提取时会造成位置信息的损失，the loss of local location information；

研究现状：

• The graph model methods have been widely used in segmentation problems[10]. The methodsregard the pixels as the nodes, and the dissimilarity between the nodes as the edges.The best segmentation is equivalent to the maximum cut in the graph.
• And combining the probability and graph theory, the probabilistic graphical model methods, suchas Markov random field and conditional random field, are applied to refine the semantic segmentation result[20,13]. These methods model the detected object as the nodes of a graph in the image, and by extracting the relation between the objects to improve the detection accuracy[15].
• Compared with the grid structure representation of input data in the deep convolution model, the graph model has a more flexible skip connection, so it can explore a variety of relationships amongthe nodes in the graph[18,17,9].

解决问题：

• we establish a graph node model for the image semantic segmentation problem；
• we initialize the graph model by the FCN. The graph model is established on a small size of the image withthe nodes annotation initialized by FCN[16] andthe weights of the edges initialized by the Gauss kernel function；
• 提出了 Graph-FCN 来解决语义分割问题。首次用 GCN 方法解决图像语义分割任务。Graph-FCN 可以扩大感受野，同时避免局部位置信息出现损失。实验表明，Graph-FCN 的性能优于 FCN。

2. Method

定义：

node：

• the node annotations are initialized by the FCN-16s；

• FCN-16s can get the feature map with a stride of 16 and 32, as shown in Fig. 2. The feature map with strides 32 can obtain the same size of the feature map with strides 16 by upsampling with the factor 2.
• the node annotations = FCN-16s两种output—stride的feature map的concat；然后把这些pixel的feature vector按一定顺序排列出来，就得到了node的特征矩阵，node annotation；

edge：

• In the graph model, the edge is respected by the adjacent matrix. We assume that each node connects to its nearest nodes. The connection means that the nodes annotation can be transferred by the edges in the the graph neural network.

• In the view of the influence of distance on correlation, we adopt the weight adjacent matrix A by the Gauss kernel function；

Training with Graph-FCN

利用Chebyshev多项式拟合卷积核是GCN论文中广泛应用的方法，本文中使用的方法完全来自于论文：Kipf T, Welling M. Semi-Supervised Classification with Graph Convolutional Networks[J]. arXiv: Learning, 2016. （引用：1701）

• The GCN is a form of Laplacian smoothing. When the messages propagate among the neighbor nodes, the neighbor nodes will tend to have similar features[14]. This means that the GCN can not be very deep for the oversmoothing, so we adopt a 2-layers GCN network.
• The maximum range of node message the current node received can be regarded as the receptive field in the graph.For the instance described in section 3.2, the size of receptive field is 5 × 32 × 32, which is five times than that of FCN-16s.（不懂？？？）Moreover, there is no nodes disappeared in this progress which means that there is no loss of local location information.
• In the Graph-FCN, the FCN-16s realize the nodes classification and initialization of the graph model in a small feature map. Meanwhile, the 2-layers GCN gets the classification of the nodes in the graph. We calculate the cross-entropy loss to the both outputs of these two parts. （也就是multi-task训练）
• The same as the FCN-16s model, the Graph-FCN is also end-to-end training.

3. Experiments

提升不多，感觉作用相当于一个post-processing模块。

更多内容，请关注：

公众号名称：计算机视觉与数字图像处理

微信号：cv_and_dip