Dynamic Graph Message Passing Network

Li Zhang1,    Dan Xu1,    Anurag Arnab2,    Philip H.S. Torr1

1University of Oxford,    2Google Research

CVPR 2020 Oral

pipeline picture

Modelling long-range dependencies is critical for complex scene understanding tasks such as semantic segmentation and object detection. Although CNNs have excelled in many computer vision tasks, they are still limited in capturing long-range structured relationships as they typically consist of layers of local kernels. A fully-connected graph is beneficial for such modelling, however, its computational overhead is prohibitive. We propose a dynamic graph message passing network, based on the message passing neural network framework, that significantly reduces the computational complexity compared to related works modelling a fully-connected graph. This is achieved by adaptively sampling nodes in the graph, conditioned on the input, for message passing. Based on the sampled nodes, we then dynamically predict node-dependent filter weights and the affinity matrix for propagating information between them. Using this model, we show significant improvements with respect to strong, state-of-the-art baselines on three different tasks and backbone architectures. Our approach also outperforms fully-connected graphs while using substantially fewer floating point operations and parameters.

Oral presentation (5 mins)

Short presentation (1 min)


Nodes sampling

pipeline picture

Semantic segmentation on Cityscapes

pipeline picture

Object detection on COCO

pipeline picture




author = {Zhang, Li and Xu, Dan and Arnab, Anurag and Torr, Philip H.S.},
title = {Dynamic Graph Message Passing Networks},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}


We thank Professor Andrew Zisserman for valuable discussions. This work was supported by the EPSRC grant Seebibyte EP/M013774/1, ERC grant ERC-2012-AdG 321162-HELIOS and EPSRC/MURI grant EP/N019474/1. We would also like to acknowledge the Royal Academy of Engineering.